[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:29:16 dist-test-slave-gkw7 krb5kdc[29643](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:29:16 dist-test-slave-gkw7 krb5kdc[29643](info): set up 2 sockets
Aug 11 06:29:16 dist-test-slave-gkw7 krb5kdc[29643](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260811 06:29:17.498934 29631 mini_kdc.cc:121] Time spent starting KDC: real 1.440s user 0.008s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/krb5kdc/test-user.keytab.
Aug 11 06:29:17 dist-test-slave-gkw7 krb5kdc[29643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429757, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:29:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:29:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.239.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.239.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.239.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab.
Entry for principal kudu/127.28.239.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab.
I20260811 06:29:17.710206 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:41831
--webserver_interface=127.28.239.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab
--principal=kudu/127.28.239.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:43029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.239.254:41831
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:29:18.107990 29664 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:29:18.108428 29664 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:29:18.108605 29664 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:29:18.117027 29664 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:29:18.117175 29664 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:29:18.117314 29664 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:29:18.117440 29664 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:29:18.117523 29664 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:29:18.129057 29664 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:43029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.239.254:41831
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab
--principal=kudu/127.28.239.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:41831
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.239.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:29:18.131096 29664 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:29:18.133070 29664 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:29:18.158645 29673 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:29:18.158823 29674 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:29:18.159024 29664 server_base.cc:1034] running on GCE node
W20260811 06:29:18.158846 29676 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:29:18.160732 29664 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:29:18.163209 29664 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:29:18.164451 29664 hybrid_clock.cc:648] HybridClock initialized: now 1786429758164358 us; error 88 us; skew 500 ppm
I20260811 06:29:18.164996 29664 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:29:18 dist-test-slave-gkw7 krb5kdc[29643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429758, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:29:18.170730 29664 init.cc:377] Logged in from keytab as kudu/127.28.239.254@KRBTEST.COM (short username kudu)
I20260811 06:29:18.173282 29664 webserver.cc:460] Webserver started at http://127.28.239.254:43671/ using document root <none> and password file <none>
I20260811 06:29:18.174455 29664 fs_manager.cc:359] Metadata directory not provided
I20260811 06:29:18.174618 29664 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:29:18.175081 29664 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:29:18.179497 29664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/minicluster-data/master-0/data/instance:
uuid: "acf798423d914df78cb866c04d3e75d0"
format_stamp: "Formatted at 2026-08-11 06:29:18 on dist-test-slave-gkw7"
server_key: "268340ba05cd50e975309cf89fde778d"
server_key_iv: "5bfaebc989f0df700464d3751439b0f4"
server_key_version: "encryptionkey@0"
I20260811 06:29:18.180620 29664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/minicluster-data/master-0/wal/instance:
uuid: "acf798423d914df78cb866c04d3e75d0"
format_stamp: "Formatted at 2026-08-11 06:29:18 on dist-test-slave-gkw7"
server_key: "268340ba05cd50e975309cf89fde778d"
server_key_iv: "5bfaebc989f0df700464d3751439b0f4"
server_key_version: "encryptionkey@0"
I20260811 06:29:18.189169 29664 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.006s sys 0.004s
I20260811 06:29:18.194770 29684 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:29:18.196851 29664 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.000s
I20260811 06:29:18.197152 29664 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/minicluster-data/master-0/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/minicluster-data/master-0/wal
uuid: "acf798423d914df78cb866c04d3e75d0"
format_stamp: "Formatted at 2026-08-11 06:29:18 on dist-test-slave-gkw7"
server_key: "268340ba05cd50e975309cf89fde778d"
server_key_iv: "5bfaebc989f0df700464d3751439b0f4"
server_key_version: "encryptionkey@0"
I20260811 06:29:18.197422 29664 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:29:18.217684 29664 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:29:18.223290 29664 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:29:18.223696 29664 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:29:18.248608 29664 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.254:41831
I20260811 06:29:18.248654 29778 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.254:41831 every 8 connection(s)
I20260811 06:29:18.251008 29664 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
I20260811 06:29:18.255740 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 29664
I20260811 06:29:18.256140 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/minicluster-data/master-0/wal/instance
I20260811 06:29:18.258199 29782 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:29:18.258529 29631 external_mini_cluster.cc:1428] Setting key 0ca96a902fe77ac35f1ab6d2b5f45da7
I20260811 06:29:18.278307 29782 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P acf798423d914df78cb866c04d3e75d0: Bootstrap starting.
I20260811 06:29:18.285189 29782 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P acf798423d914df78cb866c04d3e75d0: Neither blocks nor log segments found. Creating new log.
I20260811 06:29:18.287169 29782 log.cc:824] T 00000000000000000000000000000000 P acf798423d914df78cb866c04d3e75d0: Log is configured to *not* fsync() on all Append() calls
I20260811 06:29:18.291939 29782 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P acf798423d914df78cb866c04d3e75d0: No bootstrap required, opened a new log
Aug 11 06:29:18 dist-test-slave-gkw7 krb5kdc[29643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429757, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:29:18.305799 29782 raft_consensus.cc:348] T 00000000000000000000000000000000 P acf798423d914df78cb866c04d3e75d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acf798423d914df78cb866c04d3e75d0" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 41831 } }
I20260811 06:29:18.306496 29782 raft_consensus.cc:374] T 00000000000000000000000000000000 P acf798423d914df78cb866c04d3e75d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:29:18.306740 29782 raft_consensus.cc:729] T 00000000000000000000000000000000 P acf798423d914df78cb866c04d3e75d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acf798423d914df78cb866c04d3e75d0, State: Initialized, Role: FOLLOWER
I20260811 06:29:18.308163 29782 consensus_queue.cc:260] T 00000000000000000000000000000000 P acf798423d914df78cb866c04d3e75d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acf798423d914df78cb866c04d3e75d0" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 41831 } }
I20260811 06:29:18.308595 29782 raft_consensus.cc:388] T 00000000000000000000000000000000 P acf798423d914df78cb866c04d3e75d0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:29:18.308900 29782 raft_consensus.cc:482] T 00000000000000000000000000000000 P acf798423d914df78cb866c04d3e75d0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:29:18.309139 29782 raft_consensus.cc:3037] T 00000000000000000000000000000000 P acf798423d914df78cb866c04d3e75d0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:29:18.309135 29788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:18.266076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39748 (local address 127.28.239.254:41831)
0811 06:29:18.268733 (+ 2657us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:18.268752 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:18.268821 (+ 69us) server_negotiation.cc:407] Connection header received
0811 06:29:18.278995 (+ 10174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:18.279036 (+ 41us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:18.279892 (+ 856us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:18.280519 (+ 627us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:18.283787 (+ 3268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:18.286456 (+ 2669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:18.288846 (+ 2390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:18.290135 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:18.290450 (+ 315us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:18.296462 (+ 6012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:29:18.296558 (+ 96us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:29:18.296619 (+ 61us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:29:18.296695 (+ 76us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:29:18.301813 (+ 5118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:18.303058 (+ 1245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:18.303071 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:18.303080 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:18.303443 (+ 363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:18.303986 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:18.303992 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:18.303997 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:18.305220 (+ 1223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:29:18.305912 (+ 692us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:18.306784 (+ 872us) server_negotiation.cc:299] Negotiation successful
0811 06:29:18.307254 (+ 470us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2292,"thread_start_us":1953,"threads_started":1}
I20260811 06:29:18.312758 29782 raft_consensus.cc:504] T 00000000000000000000000000000000 P acf798423d914df78cb866c04d3e75d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acf798423d914df78cb866c04d3e75d0" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 41831 } }
I20260811 06:29:18.313794 29782 leader_election.cc:302] T 00000000000000000000000000000000 P acf798423d914df78cb866c04d3e75d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: acf798423d914df78cb866c04d3e75d0; no voters:
I20260811 06:29:18.314752 29782 leader_election.cc:288] T 00000000000000000000000000000000 P acf798423d914df78cb866c04d3e75d0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:29:18.315057 29796 raft_consensus.cc:2781] T 00000000000000000000000000000000 P acf798423d914df78cb866c04d3e75d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:29:18.316507 29796 raft_consensus.cc:686] T 00000000000000000000000000000000 P acf798423d914df78cb866c04d3e75d0 [term 1 LEADER]: Becoming Leader. State: Replica: acf798423d914df78cb866c04d3e75d0, State: Running, Role: LEADER
I20260811 06:29:18.317233 29796 consensus_queue.cc:237] T 00000000000000000000000000000000 P acf798423d914df78cb866c04d3e75d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acf798423d914df78cb866c04d3e75d0" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 41831 } }
I20260811 06:29:18.318038 29782 sys_catalog.cc:564] T 00000000000000000000000000000000 P acf798423d914df78cb866c04d3e75d0 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:29:18.324090 29798 sys_catalog.cc:455] T 00000000000000000000000000000000 P acf798423d914df78cb866c04d3e75d0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader acf798423d914df78cb866c04d3e75d0. Latest consensus state: current_term: 1 leader_uuid: "acf798423d914df78cb866c04d3e75d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acf798423d914df78cb866c04d3e75d0" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 41831 } } }
I20260811 06:29:18.324065 29797 sys_catalog.cc:455] T 00000000000000000000000000000000 P acf798423d914df78cb866c04d3e75d0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "acf798423d914df78cb866c04d3e75d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acf798423d914df78cb866c04d3e75d0" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 41831 } } }
I20260811 06:29:18.324564 29798 sys_catalog.cc:458] T 00000000000000000000000000000000 P acf798423d914df78cb866c04d3e75d0 [sys.catalog]: This master's current role is: LEADER
I20260811 06:29:18.324599 29797 sys_catalog.cc:458] T 00000000000000000000000000000000 P acf798423d914df78cb866c04d3e75d0 [sys.catalog]: This master's current role is: LEADER
I20260811 06:29:18.326177 29809 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:29:18.343757 29809 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:29:18.358422 29809 catalog_manager.cc:1347] Generated new cluster ID: b52a891c620443a38418f3e1946051e0
I20260811 06:29:18.358556 29809 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:29:18.379940 29809 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:29:18.381750 29809 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:29:18.409214 29809 catalog_manager.cc:5868] T 00000000000000000000000000000000 P acf798423d914df78cb866c04d3e75d0: Generated new TSK 0
I20260811 06:29:18.410437 29809 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.239.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.239.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.239.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.193.keytab.
Entry for principal kudu/127.28.239.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.193.keytab.
I20260811 06:29:18.468974 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.193:0
--local_ip_for_outbound_sockets=127.28.239.193
--webserver_interface=127.28.239.193
--webserver_port=0
--tserver_master_addrs=127.28.239.254:41831
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.193.keytab
--principal=kudu/127.28.239.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:43029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:29:18.821887 29837 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:29:18.822234 29837 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:29:18.822410 29837 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:29:18.829844 29837 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:29:18.829993 29837 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:29:18.830163 29837 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.193
I20260811 06:29:18.840227 29837 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:43029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.193.keytab
--principal=kudu/127.28.239.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.239.193
--webserver_port=0
--tserver_master_addrs=127.28.239.254:41831
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.193
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:29:18.841658 29837 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:29:18.843078 29837 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:29:18.857939 29847 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:29:18.857969 29852 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:29:18.857908 29854 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:29:18.858784 29837 server_base.cc:1034] running on GCE node
I20260811 06:29:18.859478 29837 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:29:18.860450 29837 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:29:18.861624 29837 hybrid_clock.cc:648] HybridClock initialized: now 1786429758861560 us; error 60 us; skew 500 ppm
I20260811 06:29:18.861969 29837 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:29:18 dist-test-slave-gkw7 krb5kdc[29643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429758, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:29:18.865279 29837 init.cc:377] Logged in from keytab as kudu/127.28.239.193@KRBTEST.COM (short username kudu)
I20260811 06:29:18.866915 29837 webserver.cc:460] Webserver started at http://127.28.239.193:33509/ using document root <none> and password file <none>
I20260811 06:29:18.867627 29837 fs_manager.cc:359] Metadata directory not provided
I20260811 06:29:18.867779 29837 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:29:18.868103 29837 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:29:18.871284 29837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/minicluster-data/ts-0/data/instance:
uuid: "549685fb6c7a45dfa55557e50a7d50d3"
format_stamp: "Formatted at 2026-08-11 06:29:18 on dist-test-slave-gkw7"
server_key: "1e0a8f549040f7f91edec94f00befb92"
server_key_iv: "118d07f4cf9376c4668ceaa80df2453c"
server_key_version: "encryptionkey@0"
I20260811 06:29:18.872051 29837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/minicluster-data/ts-0/wal/instance:
uuid: "549685fb6c7a45dfa55557e50a7d50d3"
format_stamp: "Formatted at 2026-08-11 06:29:18 on dist-test-slave-gkw7"
server_key: "1e0a8f549040f7f91edec94f00befb92"
server_key_iv: "118d07f4cf9376c4668ceaa80df2453c"
server_key_version: "encryptionkey@0"
I20260811 06:29:18.877324 29837 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 06:29:18.880852 29864 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:29:18.882263 29837 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:29:18.882440 29837 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/minicluster-data/ts-0/wal
uuid: "549685fb6c7a45dfa55557e50a7d50d3"
format_stamp: "Formatted at 2026-08-11 06:29:18 on dist-test-slave-gkw7"
server_key: "1e0a8f549040f7f91edec94f00befb92"
server_key_iv: "118d07f4cf9376c4668ceaa80df2453c"
server_key_version: "encryptionkey@0"
I20260811 06:29:18.882639 29837 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:29:18.922524 29837 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:29:18.926743 29837 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:29:18.927086 29837 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:29:18.928244 29837 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:29:18.930070 29837 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:29:18.930173 29837 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:29:18.930377 29837 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:29:18.930459 29837 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:29:18.966374 29837 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.193:37669
I20260811 06:29:18.966431 30047 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.193:37669 every 8 connection(s)
I20260811 06:29:18.967722 29837 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
I20260811 06:29:18.970715 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 29837
I20260811 06:29:18.970948 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/minicluster-data/ts-0/wal/instance
I20260811 06:29:18.971673 29631 external_mini_cluster.cc:1428] Setting key 3420a57eba6addd334f4e3652a94d1b8
Aug 11 06:29:18 dist-test-slave-gkw7 krb5kdc[29643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429758, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.193@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:29:18.995740 30050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:18.971560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.193:45649 (local address 127.28.239.254:41831)
0811 06:29:18.972089 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:18.972097 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:18.973251 (+ 1154us) server_negotiation.cc:407] Connection header received
0811 06:29:18.976084 (+ 2833us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:18.976088 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:18.976157 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:18.976254 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:18.978866 (+ 2612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:18.980027 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:18.982941 (+ 2914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:18.983388 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:18.983533 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:18.988247 (+ 4714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:29:18.988255 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:29:18.988260 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:29:18.988303 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:29:18.991802 (+ 3499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:18.992870 (+ 1068us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:18.992876 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:18.992880 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:18.993035 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:18.993633 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:18.993639 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:18.993642 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:18.994078 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:29:18.994255 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:18.995104 (+ 849us) server_negotiation.cc:299] Negotiation successful
0811 06:29:18.995361 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396,"thread_start_us":291,"threads_started":1}
I20260811 06:29:18.997371 30051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:18.972211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:41831 (local address 127.28.239.193:45649)
0811 06:29:18.973106 (+ 895us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:18.973148 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 06:29:18.975760 (+ 2612us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:29:18.976672 (+ 912us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:29:18.976688 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:29:18.977599 (+ 911us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:29:18.978659 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:18.978689 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:18.980224 (+ 1535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:18.980232 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:18.982781 (+ 2549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:18.982795 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:18.983654 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:18.983665 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:18.983938 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:18.984648 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:29:18.984672 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:29:18.987965 (+ 3293us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:29:18.992012 (+ 4047us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:29:18.992023 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:29:18.992045 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:29:18.992732 (+ 687us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:29:18.993165 (+ 433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:29:18.993171 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:29:18.993175 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:29:18.993450 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:29:18.994249 (+ 799us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:29:18.994258 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:29:18.994646 (+ 388us) client_negotiation.cc:769] Sending connection context
0811 06:29:18.994993 (+ 347us) client_negotiation.cc:240] Negotiation successful
0811 06:29:18.995577 (+ 584us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":628,"thread_start_us":403,"threads_started":1}
WARNING: no policy specified for kudu/127.28.239.194@KRBTEST.COM; defaulting to no policy
I20260811 06:29:18.999666 30048 heartbeater.cc:347] Connected to a master server at 127.28.239.254:41831
I20260811 06:29:19.000337 30048 heartbeater.cc:464] Registering TS with master...
I20260811 06:29:19.001554 30048 heartbeater.cc:511] Master 127.28.239.254:41831 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.239.194@KRBTEST.COM" created.
I20260811 06:29:19.006575 29711 ts_manager.cc:194] Registered new tserver with Master: 549685fb6c7a45dfa55557e50a7d50d3 (127.28.239.193:37669)
I20260811 06:29:19.009903 29711 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.239.193@KRBTEST.COM'} at 127.28.239.193:45649
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.239.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.194.keytab.
Entry for principal kudu/127.28.239.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.194.keytab.
I20260811 06:29:19.038234 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.194:0
--local_ip_for_outbound_sockets=127.28.239.194
--webserver_interface=127.28.239.194
--webserver_port=0
--tserver_master_addrs=127.28.239.254:41831
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.194.keytab
--principal=kudu/127.28.239.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:43029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:29:19.396564 30058 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:29:19.396857 30058 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:29:19.397024 30058 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:29:19.404584 30058 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:29:19.404700 30058 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:29:19.404906 30058 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.194
I20260811 06:29:19.414997 30058 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:43029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.194.keytab
--principal=kudu/127.28.239.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.239.194
--webserver_port=0
--tserver_master_addrs=127.28.239.254:41831
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.194
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:29:19.416327 30058 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:29:19.417841 30058 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:29:19.433049 30064 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:29:19.433018 30067 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:29:19.432955 30070 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:29:19.433950 30058 server_base.cc:1034] running on GCE node
I20260811 06:29:19.434582 30058 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:29:19.435669 30058 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:29:19.436879 30058 hybrid_clock.cc:648] HybridClock initialized: now 1786429759436829 us; error 43 us; skew 500 ppm
I20260811 06:29:19.437222 30058 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:29:19 dist-test-slave-gkw7 krb5kdc[29643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429759, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:29:19.440555 30058 init.cc:377] Logged in from keytab as kudu/127.28.239.194@KRBTEST.COM (short username kudu)
I20260811 06:29:19.442356 30058 webserver.cc:460] Webserver started at http://127.28.239.194:38355/ using document root <none> and password file <none>
I20260811 06:29:19.443096 30058 fs_manager.cc:359] Metadata directory not provided
I20260811 06:29:19.443214 30058 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:29:19.443524 30058 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:29:19.446746 30058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/minicluster-data/ts-1/data/instance:
uuid: "f662ce73366344488b8d7f3b76afb784"
format_stamp: "Formatted at 2026-08-11 06:29:19 on dist-test-slave-gkw7"
server_key: "f62d0be99c079e3523140a04eaae2fa5"
server_key_iv: "895bbf2c5fc8fe2de84c21a5632e4da4"
server_key_version: "encryptionkey@0"
I20260811 06:29:19.447551 30058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/minicluster-data/ts-1/wal/instance:
uuid: "f662ce73366344488b8d7f3b76afb784"
format_stamp: "Formatted at 2026-08-11 06:29:19 on dist-test-slave-gkw7"
server_key: "f62d0be99c079e3523140a04eaae2fa5"
server_key_iv: "895bbf2c5fc8fe2de84c21a5632e4da4"
server_key_version: "encryptionkey@0"
I20260811 06:29:19.453784 30058 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260811 06:29:19.457578 30080 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:29:19.459049 30058 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 06:29:19.459240 30058 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/minicluster-data/ts-1/wal
uuid: "f662ce73366344488b8d7f3b76afb784"
format_stamp: "Formatted at 2026-08-11 06:29:19 on dist-test-slave-gkw7"
server_key: "f62d0be99c079e3523140a04eaae2fa5"
server_key_iv: "895bbf2c5fc8fe2de84c21a5632e4da4"
server_key_version: "encryptionkey@0"
I20260811 06:29:19.459460 30058 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:29:19.474941 30058 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:29:19.479430 30058 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:29:19.479740 30058 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:29:19.480751 30058 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:29:19.482199 30058 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:29:19.482273 30058 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:29:19.482399 30058 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:29:19.482455 30058 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:29:19.521678 30058 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.194:40117
I20260811 06:29:19.521729 30271 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.194:40117 every 8 connection(s)
I20260811 06:29:19.523198 30058 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
I20260811 06:29:19.531946 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 30058
I20260811 06:29:19.532157 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/minicluster-data/ts-1/wal/instance
I20260811 06:29:19.532593 29631 external_mini_cluster.cc:1428] Setting key dc0721c3b62db41f093e202ec084058f
Aug 11 06:29:19 dist-test-slave-gkw7 krb5kdc[29643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429759, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.194@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:29:19.546814 30276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:19.526045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.194:42879 (local address 127.28.239.254:41831)
0811 06:29:19.526543 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:19.526547 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:19.527669 (+ 1122us) server_negotiation.cc:407] Connection header received
0811 06:29:19.529769 (+ 2100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:19.529775 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:19.529878 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:19.530021 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:19.531677 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:19.532599 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:19.535113 (+ 2514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:19.535656 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:19.535796 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:19.539657 (+ 3861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:29:19.539666 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:29:19.539674 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:29:19.539723 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:29:19.542864 (+ 3141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:19.543965 (+ 1101us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:19.543969 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:19.543972 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:19.544112 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:19.544680 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:19.544684 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:19.544686 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:19.544981 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:29:19.545099 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:19.546348 (+ 1249us) server_negotiation.cc:299] Negotiation successful
0811 06:29:19.546527 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"thread_start_us":271,"threads_started":1}
I20260811 06:29:19.548523 30277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:19.526701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:41831 (local address 127.28.239.194:42879)
0811 06:29:19.527473 (+ 772us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:19.527526 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 06:29:19.529413 (+ 1887us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:29:19.530285 (+ 872us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:29:19.530294 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:29:19.530866 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:29:19.531484 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:19.531502 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:19.532803 (+ 1301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:19.532808 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:19.534944 (+ 2136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:19.534960 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:19.535913 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:19.535922 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:19.536191 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:19.536920 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:29:19.536944 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:29:19.539414 (+ 2470us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:29:19.543020 (+ 3606us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:29:19.543034 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:29:19.543055 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:29:19.543822 (+ 767us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:29:19.544239 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:29:19.544248 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:29:19.544255 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:29:19.544537 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:29:19.545142 (+ 605us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:29:19.545155 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:29:19.545874 (+ 719us) client_negotiation.cc:769] Sending connection context
0811 06:29:19.546266 (+ 392us) client_negotiation.cc:240] Negotiation successful
0811 06:29:19.546879 (+ 613us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":292,"threads_started":1}
I20260811 06:29:19.550354 30273 heartbeater.cc:347] Connected to a master server at 127.28.239.254:41831
I20260811 06:29:19.550727 30273 heartbeater.cc:464] Registering TS with master...
I20260811 06:29:19.551730 30273 heartbeater.cc:511] Master 127.28.239.254:41831 requested a full tablet report, sending...
I20260811 06:29:19.553498 29711 ts_manager.cc:194] Registered new tserver with Master: f662ce73366344488b8d7f3b76afb784 (127.28.239.194:40117)
WARNING: no policy specified for kudu/127.28.239.195@KRBTEST.COM; defaulting to no policy
I20260811 06:29:19.554690 29711 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.239.194@KRBTEST.COM'} at 127.28.239.194:42879
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.239.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.239.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.195.keytab.
Entry for principal kudu/127.28.239.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.195.keytab.
I20260811 06:29:19.582119 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.195:0
--local_ip_for_outbound_sockets=127.28.239.195
--webserver_interface=127.28.239.195
--webserver_port=0
--tserver_master_addrs=127.28.239.254:41831
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.195.keytab
--principal=kudu/127.28.239.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:43029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:29:19.928750 30282 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:29:19.929078 30282 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:29:19.929255 30282 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:29:19.936615 30282 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:29:19.936759 30282 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:29:19.936937 30282 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.195
I20260811 06:29:19.946954 30282 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:43029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.195.keytab
--principal=kudu/127.28.239.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.239.195
--webserver_port=0
--tserver_master_addrs=127.28.239.254:41831
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.195
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:29:19.948300 30282 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:29:19.949851 30282 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:29:19.963385 30294 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:29:19.963415 30295 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:29:19.963367 30297 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:29:19.964816 30282 server_base.cc:1034] running on GCE node
I20260811 06:29:19.965507 30282 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:29:19.966730 30282 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:29:19.967947 30282 hybrid_clock.cc:648] HybridClock initialized: now 1786429759967884 us; error 53 us; skew 500 ppm
I20260811 06:29:19.968328 30282 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:29:19 dist-test-slave-gkw7 krb5kdc[29643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429759, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:29:19.971961 30282 init.cc:377] Logged in from keytab as kudu/127.28.239.195@KRBTEST.COM (short username kudu)
I20260811 06:29:19.973625 30282 webserver.cc:460] Webserver started at http://127.28.239.195:42399/ using document root <none> and password file <none>
I20260811 06:29:19.974372 30282 fs_manager.cc:359] Metadata directory not provided
I20260811 06:29:19.974486 30282 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:29:19.974787 30282 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:29:19.978125 30282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/minicluster-data/ts-2/data/instance:
uuid: "3432a71ef2944252b98cd6564f029309"
format_stamp: "Formatted at 2026-08-11 06:29:19 on dist-test-slave-gkw7"
server_key: "021920ed27966ce741a2a607ed4c0115"
server_key_iv: "dcce4c32ee3c0796fe3112b4212b4600"
server_key_version: "encryptionkey@0"
I20260811 06:29:19.978917 30282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/minicluster-data/ts-2/wal/instance:
uuid: "3432a71ef2944252b98cd6564f029309"
format_stamp: "Formatted at 2026-08-11 06:29:19 on dist-test-slave-gkw7"
server_key: "021920ed27966ce741a2a607ed4c0115"
server_key_iv: "dcce4c32ee3c0796fe3112b4212b4600"
server_key_version: "encryptionkey@0"
I20260811 06:29:19.984696 30282 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260811 06:29:19.988554 30304 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:29:19.990007 30282 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:29:19.990157 30282 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/minicluster-data/ts-2/wal
uuid: "3432a71ef2944252b98cd6564f029309"
format_stamp: "Formatted at 2026-08-11 06:29:19 on dist-test-slave-gkw7"
server_key: "021920ed27966ce741a2a607ed4c0115"
server_key_iv: "dcce4c32ee3c0796fe3112b4212b4600"
server_key_version: "encryptionkey@0"
I20260811 06:29:19.990340 30282 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:29:20.015404 30048 heartbeater.cc:503] Master 127.28.239.254:41831 was elected leader, sending a full tablet report...
I20260811 06:29:20.038733 30282 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:29:20.044512 30282 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:29:20.044865 30282 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:29:20.046041 30282 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:29:20.047497 30282 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:29:20.047577 30282 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:29:20.047753 30282 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:29:20.047829 30282 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:29:20.085434 30282 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.195:37499
I20260811 06:29:20.085475 30493 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.195:37499 every 8 connection(s)
I20260811 06:29:20.087004 30282 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
I20260811 06:29:20.092409 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 30282
I20260811 06:29:20.092617 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786429755927508-29631-0/minicluster-data/ts-2/wal/instance
I20260811 06:29:20.093186 29631 external_mini_cluster.cc:1428] Setting key 28330ac70dbc46cd6b888c2dc7662b3f
Aug 11 06:29:20 dist-test-slave-gkw7 krb5kdc[29643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429759, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.195@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:29:20.109480 30504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:20.089844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.195:53891 (local address 127.28.239.254:41831)
0811 06:29:20.090304 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:20.090308 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:20.091325 (+ 1017us) server_negotiation.cc:407] Connection header received
0811 06:29:20.094047 (+ 2722us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:20.094052 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:20.094136 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:20.094316 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:20.096129 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:20.097210 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:20.099059 (+ 1849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:20.099498 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:20.099633 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:20.103346 (+ 3713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:29:20.103351 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:29:20.103354 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:29:20.103388 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:29:20.106272 (+ 2884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:20.107007 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:20.107011 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:20.107013 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:20.107187 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:20.107580 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:20.107584 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:20.107586 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:20.107878 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:29:20.107999 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:20.108887 (+ 888us) server_negotiation.cc:299] Negotiation successful
0811 06:29:20.109146 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":264,"threads_started":1}
I20260811 06:29:20.110648 30505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:20.090416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:41831 (local address 127.28.239.195:53891)
0811 06:29:20.091114 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:20.091188 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0811 06:29:20.093676 (+ 2488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:29:20.094581 (+ 905us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:29:20.094590 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:29:20.095334 (+ 744us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:29:20.095943 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:20.095962 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:20.097373 (+ 1411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:20.097378 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:20.098906 (+ 1528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:20.098918 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:20.099714 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:20.099719 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:20.099915 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:20.100647 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:29:20.100669 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:29:20.103146 (+ 2477us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:29:20.106420 (+ 3274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:29:20.106427 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:29:20.106440 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:29:20.106883 (+ 443us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:29:20.107285 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:29:20.107288 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:29:20.107290 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:29:20.107476 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:29:20.107980 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:29:20.107986 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:29:20.108428 (+ 442us) client_negotiation.cc:769] Sending connection context
0811 06:29:20.108742 (+ 314us) client_negotiation.cc:240] Negotiation successful
0811 06:29:20.109286 (+ 544us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":286,"threads_started":1}
I20260811 06:29:20.111876 30497 heartbeater.cc:347] Connected to a master server at 127.28.239.254:41831
I20260811 06:29:20.112139 30497 heartbeater.cc:464] Registering TS with master...
I20260811 06:29:20.112803 30497 heartbeater.cc:511] Master 127.28.239.254:41831 requested a full tablet report, sending...
I20260811 06:29:20.114151 29711 ts_manager.cc:194] Registered new tserver with Master: 3432a71ef2944252b98cd6564f029309 (127.28.239.195:37499)
I20260811 06:29:20.114919 29711 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.239.195@KRBTEST.COM'} at 127.28.239.195:53891
I20260811 06:29:20.122655 29631 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 06:29:20 dist-test-slave-gkw7 krb5kdc[29643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429760, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 06:29:20 dist-test-slave-gkw7 krb5kdc[29643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429760, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.28.239.193@KRBTEST.COM
I20260811 06:29:20.172806 30512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:20.156949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44974 (local address 127.28.239.193:37669)
0811 06:29:20.158183 (+ 1234us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:20.158190 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:20.158219 (+ 29us) server_negotiation.cc:407] Connection header received
0811 06:29:20.158312 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:20.158319 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:20.158531 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:20.158677 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:20.160066 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:20.161105 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:20.162455 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:20.163023 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:20.163187 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:20.167421 (+ 4234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:29:20.167432 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:29:20.167447 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:29:20.167484 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:29:20.170272 (+ 2788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:20.170881 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:20.170888 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:20.170890 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:20.171008 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:20.171448 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:20.171452 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:20.171454 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:20.171805 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:29:20.172048 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:20.172325 (+ 277us) server_negotiation.cc:299] Negotiation successful
0811 06:29:20.172525 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1078,"thread_start_us":898,"threads_started":1}
W20260811 06:29:20.174263 29908 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44974
Aug 11 06:29:20 dist-test-slave-gkw7 krb5kdc[29643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429760, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 06:29:20 dist-test-slave-gkw7 krb5kdc[29643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429760, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.239.193@KRBTEST.COM
I20260811 06:29:20.222885 30512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:20.208547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44980 (local address 127.28.239.193:37669)
0811 06:29:20.208804 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:20.208813 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:20.209043 (+ 230us) server_negotiation.cc:407] Connection header received
0811 06:29:20.209286 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:20.209290 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:20.209371 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:20.209483 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:20.210904 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:20.211656 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:20.212908 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:20.213369 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:20.213503 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:20.217355 (+ 3852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:29:20.217360 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:29:20.217363 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:29:20.217396 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:29:20.220230 (+ 2834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:20.220854 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:20.220861 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:20.220864 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:20.220995 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:20.221382 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:20.221388 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:20.221391 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:20.221709 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:29:20.221972 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:20.222320 (+ 348us) server_negotiation.cc:299] Negotiation successful
0811 06:29:20.222553 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260811 06:29:20.224818 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 29837
I20260811 06:29:20.238459 30512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:20.225930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44994 (local address 127.28.239.193:37669)
0811 06:29:20.226081 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:20.226085 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:20.226443 (+ 358us) server_negotiation.cc:407] Connection header received
0811 06:29:20.226559 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:20.226563 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:20.226616 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:20.226694 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:20.227788 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:20.228462 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:20.229789 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:20.230189 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:20.230282 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:20.232570 (+ 2288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:29:20.232575 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:29:20.232578 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:29:20.232606 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:29:20.235196 (+ 2590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:20.235919 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:20.235923 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:20.235926 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:20.236104 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:20.236670 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:20.236677 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:20.236680 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:20.237046 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:29:20.237306 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:20.237798 (+ 492us) server_negotiation.cc:299] Negotiation successful
0811 06:29:20.238028 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 06:29:20.239652 30042 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:29:20.333307 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 29837
I20260811 06:29:20.359552 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 30058
Aug 11 06:29:20 dist-test-slave-gkw7 krb5kdc[29643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429760, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.239.194@KRBTEST.COM
I20260811 06:29:20.375751 30523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:20.360482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42726 (local address 127.28.239.194:40117)
0811 06:29:20.361108 (+ 626us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:20.361115 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:20.361144 (+ 29us) server_negotiation.cc:407] Connection header received
0811 06:29:20.361209 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:20.361215 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:20.361437 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:20.361651 (+ 214us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:20.362757 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:20.363961 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:20.365234 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:20.365967 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:20.366065 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:20.370058 (+ 3993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:29:20.370069 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:29:20.370084 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:29:20.370115 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:29:20.372886 (+ 2771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:20.373604 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:20.373617 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:20.373629 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:20.373796 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:20.374262 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:20.374266 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:20.374268 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:20.374612 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:29:20.374851 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:20.375133 (+ 282us) server_negotiation.cc:299] Negotiation successful
0811 06:29:20.375328 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":476,"thread_start_us":332,"threads_started":1}
I20260811 06:29:20.377206 30267 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:29:20.460503 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 30058
I20260811 06:29:20.486526 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 30282
Aug 11 06:29:20 dist-test-slave-gkw7 krb5kdc[29643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429760, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.239.195@KRBTEST.COM
I20260811 06:29:20.503906 30526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:20.487663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44266 (local address 127.28.239.195:37499)
0811 06:29:20.488340 (+ 677us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:20.488348 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:20.488381 (+ 33us) server_negotiation.cc:407] Connection header received
0811 06:29:20.488447 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:20.488454 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:20.488705 (+ 251us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:20.488848 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:20.490146 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:20.491292 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:20.492593 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:20.493195 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:20.493315 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:20.497917 (+ 4602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:29:20.497928 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:29:20.497944 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:29:20.497979 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:29:20.500797 (+ 2818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:20.501504 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:20.501512 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:20.501514 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:20.501731 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:20.502162 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:20.502165 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:20.502167 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:20.502591 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:29:20.502856 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:20.503306 (+ 450us) server_negotiation.cc:299] Negotiation successful
0811 06:29:20.503511 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":511,"thread_start_us":351,"threads_started":1}
I20260811 06:29:20.505236 30482 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:29:20.590015 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 30282
I20260811 06:29:20.613915 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 29664
I20260811 06:29:20.615207 29765 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:29:20.721601 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 29664
2026-08-11T06:29:20Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (4729 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:29:20 dist-test-slave-gkw7 krb5kdc[30534](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:29:20 dist-test-slave-gkw7 krb5kdc[30534](info): set up 2 sockets
Aug 11 06:29:20 dist-test-slave-gkw7 krb5kdc[30534](info): commencing operation
krb5kdc: starting...
W20260811 06:29:22.067510 29631 mini_kdc.cc:121] Time spent starting KDC: real 1.294s user 0.003s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/krb5kdc/test-user.keytab.
Aug 11 06:29:22 dist-test-slave-gkw7 krb5kdc[30534](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429762, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:29:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:29:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.239.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.239.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.239.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab.
Entry for principal kudu/127.28.239.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab.
I20260811 06:29:22.277951 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:40417
--webserver_interface=127.28.239.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab
--principal=kudu/127.28.239.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:43819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.239.254:40417
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:29:22.616698 30549 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:29:22.616993 30549 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:29:22.617089 30549 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:29:22.624584 30549 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:29:22.624711 30549 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:29:22.624759 30549 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:29:22.624828 30549 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:29:22.624873 30549 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:29:22.634857 30549 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:43819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.239.254:40417
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab
--principal=kudu/127.28.239.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:40417
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.239.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:29:22.636286 30549 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:29:22.637584 30549 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:29:22.649978 30556 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:29:22.649860 30560 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:29:22.649941 30558 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:29:22.651113 30549 server_base.cc:1034] running on GCE node
I20260811 06:29:22.651897 30549 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:29:22.653306 30549 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:29:22.654520 30549 hybrid_clock.cc:648] HybridClock initialized: now 1786429762654433 us; error 72 us; skew 500 ppm
I20260811 06:29:22.654848 30549 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:29:22 dist-test-slave-gkw7 krb5kdc[30534](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429762, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:29:22.663324 30549 init.cc:377] Logged in from keytab as kudu/127.28.239.254@KRBTEST.COM (short username kudu)
I20260811 06:29:22.664969 30549 webserver.cc:460] Webserver started at http://127.28.239.254:37567/ using document root <none> and password file <none>
I20260811 06:29:22.665764 30549 fs_manager.cc:359] Metadata directory not provided
I20260811 06:29:22.665902 30549 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:29:22.666244 30549 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:29:22.669296 30549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/minicluster-data/master-0/data/instance:
uuid: "54fe84d2af6649c583bad00f6c1546de"
format_stamp: "Formatted at 2026-08-11 06:29:22 on dist-test-slave-gkw7"
server_key: "db18081605b60dd92bdce983e092aaa9"
server_key_iv: "f1e209c775816bb9ba9ccce83d1687f0"
server_key_version: "encryptionkey@0"
I20260811 06:29:22.670174 30549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/minicluster-data/master-0/wal/instance:
uuid: "54fe84d2af6649c583bad00f6c1546de"
format_stamp: "Formatted at 2026-08-11 06:29:22 on dist-test-slave-gkw7"
server_key: "db18081605b60dd92bdce983e092aaa9"
server_key_iv: "f1e209c775816bb9ba9ccce83d1687f0"
server_key_version: "encryptionkey@0"
I20260811 06:29:22.675590 30549 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 06:29:22.679330 30572 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:29:22.680826 30549 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 06:29:22.681026 30549 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/minicluster-data/master-0/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/minicluster-data/master-0/wal
uuid: "54fe84d2af6649c583bad00f6c1546de"
format_stamp: "Formatted at 2026-08-11 06:29:22 on dist-test-slave-gkw7"
server_key: "db18081605b60dd92bdce983e092aaa9"
server_key_iv: "f1e209c775816bb9ba9ccce83d1687f0"
server_key_version: "encryptionkey@0"
I20260811 06:29:22.681231 30549 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:29:22.711304 30549 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:29:22.715770 30549 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:29:22.716082 30549 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:29:22.735661 30549 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.254:40417
I20260811 06:29:22.735703 30667 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.254:40417 every 8 connection(s)
I20260811 06:29:22.737094 30549 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
I20260811 06:29:22.738267 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 30549
I20260811 06:29:22.738547 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/minicluster-data/master-0/wal/instance
I20260811 06:29:22.739425 29631 external_mini_cluster.cc:1428] Setting key f132223c2f9c27f301f6c3a9cab88083
I20260811 06:29:22.742308 30668 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 06:29:22 dist-test-slave-gkw7 krb5kdc[30534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429762, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:29:22.755909 30668 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54fe84d2af6649c583bad00f6c1546de: Bootstrap starting.
I20260811 06:29:22.760226 30668 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 54fe84d2af6649c583bad00f6c1546de: Neither blocks nor log segments found. Creating new log.
I20260811 06:29:22.761652 30668 log.cc:824] T 00000000000000000000000000000000 P 54fe84d2af6649c583bad00f6c1546de: Log is configured to *not* fsync() on all Append() calls
I20260811 06:29:22.764163 30674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:22.741703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56460 (local address 127.28.239.254:40417)
0811 06:29:22.742598 (+ 895us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:22.742610 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:22.742659 (+ 49us) server_negotiation.cc:407] Connection header received
0811 06:29:22.744271 (+ 1612us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:22.744293 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:22.744694 (+ 401us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:22.745031 (+ 337us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:22.746983 (+ 1952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:22.748360 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:22.750121 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:22.750792 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:22.750924 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:22.755764 (+ 4840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:29:22.755775 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:29:22.755795 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:29:22.755841 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:29:22.759838 (+ 3997us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:22.760552 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:22.760561 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:22.760573 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:22.760808 (+ 235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:22.761153 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:22.761159 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:22.761162 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:22.761959 (+ 797us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:29:22.762304 (+ 345us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:22.762771 (+ 467us) server_negotiation.cc:299] Negotiation successful
0811 06:29:22.763146 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":662,"thread_start_us":385,"threads_started":1}
I20260811 06:29:22.764876 30668 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54fe84d2af6649c583bad00f6c1546de: No bootstrap required, opened a new log
I20260811 06:29:22.774955 30668 raft_consensus.cc:348] T 00000000000000000000000000000000 P 54fe84d2af6649c583bad00f6c1546de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54fe84d2af6649c583bad00f6c1546de" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 40417 } }
I20260811 06:29:22.775461 30668 raft_consensus.cc:374] T 00000000000000000000000000000000 P 54fe84d2af6649c583bad00f6c1546de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:29:22.775568 30668 raft_consensus.cc:729] T 00000000000000000000000000000000 P 54fe84d2af6649c583bad00f6c1546de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54fe84d2af6649c583bad00f6c1546de, State: Initialized, Role: FOLLOWER
I20260811 06:29:22.776328 30668 consensus_queue.cc:260] T 00000000000000000000000000000000 P 54fe84d2af6649c583bad00f6c1546de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54fe84d2af6649c583bad00f6c1546de" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 40417 } }
I20260811 06:29:22.776657 30668 raft_consensus.cc:388] T 00000000000000000000000000000000 P 54fe84d2af6649c583bad00f6c1546de [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:29:22.776782 30668 raft_consensus.cc:482] T 00000000000000000000000000000000 P 54fe84d2af6649c583bad00f6c1546de [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:29:22.776938 30668 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 54fe84d2af6649c583bad00f6c1546de [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:29:22.780544 30668 raft_consensus.cc:504] T 00000000000000000000000000000000 P 54fe84d2af6649c583bad00f6c1546de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54fe84d2af6649c583bad00f6c1546de" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 40417 } }
I20260811 06:29:22.781198 30668 leader_election.cc:302] T 00000000000000000000000000000000 P 54fe84d2af6649c583bad00f6c1546de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 54fe84d2af6649c583bad00f6c1546de; no voters:
I20260811 06:29:22.782075 30668 leader_election.cc:288] T 00000000000000000000000000000000 P 54fe84d2af6649c583bad00f6c1546de [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:29:22.782316 30676 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 54fe84d2af6649c583bad00f6c1546de [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:29:22.783057 30676 raft_consensus.cc:686] T 00000000000000000000000000000000 P 54fe84d2af6649c583bad00f6c1546de [term 1 LEADER]: Becoming Leader. State: Replica: 54fe84d2af6649c583bad00f6c1546de, State: Running, Role: LEADER
I20260811 06:29:22.783691 30676 consensus_queue.cc:237] T 00000000000000000000000000000000 P 54fe84d2af6649c583bad00f6c1546de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54fe84d2af6649c583bad00f6c1546de" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 40417 } }
I20260811 06:29:22.784541 30668 sys_catalog.cc:564] T 00000000000000000000000000000000 P 54fe84d2af6649c583bad00f6c1546de [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:29:22.787241 30677 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54fe84d2af6649c583bad00f6c1546de [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "54fe84d2af6649c583bad00f6c1546de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54fe84d2af6649c583bad00f6c1546de" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 40417 } } }
I20260811 06:29:22.787693 30677 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54fe84d2af6649c583bad00f6c1546de [sys.catalog]: This master's current role is: LEADER
I20260811 06:29:22.787217 30678 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54fe84d2af6649c583bad00f6c1546de [sys.catalog]: SysCatalogTable state changed. Reason: New leader 54fe84d2af6649c583bad00f6c1546de. Latest consensus state: current_term: 1 leader_uuid: "54fe84d2af6649c583bad00f6c1546de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54fe84d2af6649c583bad00f6c1546de" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 40417 } } }
I20260811 06:29:22.788070 30678 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54fe84d2af6649c583bad00f6c1546de [sys.catalog]: This master's current role is: LEADER
I20260811 06:29:22.789466 30686 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:29:22.795632 30686 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:29:22.805517 30686 catalog_manager.cc:1347] Generated new cluster ID: 478b9e0dde544234b9226bf67649bd26
I20260811 06:29:22.805683 30686 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:29:22.841054 30686 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:29:22.842497 30686 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:29:22.854314 30686 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 54fe84d2af6649c583bad00f6c1546de: Generated new TSK 0
I20260811 06:29:22.855209 30686 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.239.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.239.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.239.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.193.keytab.
Entry for principal kudu/127.28.239.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.193.keytab.
I20260811 06:29:22.905781 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.193:0
--local_ip_for_outbound_sockets=127.28.239.193
--webserver_interface=127.28.239.193
--webserver_port=0
--tserver_master_addrs=127.28.239.254:40417
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.193.keytab
--principal=kudu/127.28.239.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:43819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:29:23.250183 30709 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:29:23.250631 30709 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:29:23.250859 30709 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:29:23.258401 30709 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:29:23.258529 30709 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:29:23.258705 30709 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.193
I20260811 06:29:23.268636 30709 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:43819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.193.keytab
--principal=kudu/127.28.239.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.239.193
--webserver_port=0
--tserver_master_addrs=127.28.239.254:40417
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.193
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:29:23.270107 30709 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:29:23.271478 30709 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:29:23.288081 30722 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:29:23.288028 30723 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:29:23.288149 30727 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:29:23.289580 30709 server_base.cc:1034] running on GCE node
I20260811 06:29:23.290293 30709 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:29:23.291280 30709 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:29:23.292529 30709 hybrid_clock.cc:648] HybridClock initialized: now 1786429763292469 us; error 51 us; skew 500 ppm
I20260811 06:29:23.292870 30709 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:29:23 dist-test-slave-gkw7 krb5kdc[30534](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429763, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:29:23.296089 30709 init.cc:377] Logged in from keytab as kudu/127.28.239.193@KRBTEST.COM (short username kudu)
I20260811 06:29:23.297811 30709 webserver.cc:460] Webserver started at http://127.28.239.193:38695/ using document root <none> and password file <none>
I20260811 06:29:23.298554 30709 fs_manager.cc:359] Metadata directory not provided
I20260811 06:29:23.298718 30709 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:29:23.299081 30709 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:29:23.302230 30709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/minicluster-data/ts-0/data/instance:
uuid: "53f641564e96438bb6170e32d7bc91d6"
format_stamp: "Formatted at 2026-08-11 06:29:23 on dist-test-slave-gkw7"
server_key: "33892fcfdb178202551e6593d73ed280"
server_key_iv: "f2494fc9e8b2588023133a7d981d0d56"
server_key_version: "encryptionkey@0"
I20260811 06:29:23.303061 30709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/minicluster-data/ts-0/wal/instance:
uuid: "53f641564e96438bb6170e32d7bc91d6"
format_stamp: "Formatted at 2026-08-11 06:29:23 on dist-test-slave-gkw7"
server_key: "33892fcfdb178202551e6593d73ed280"
server_key_iv: "f2494fc9e8b2588023133a7d981d0d56"
server_key_version: "encryptionkey@0"
I20260811 06:29:23.308619 30709 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 06:29:23.312623 30749 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:29:23.314322 30709 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:29:23.314533 30709 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/minicluster-data/ts-0/wal
uuid: "53f641564e96438bb6170e32d7bc91d6"
format_stamp: "Formatted at 2026-08-11 06:29:23 on dist-test-slave-gkw7"
server_key: "33892fcfdb178202551e6593d73ed280"
server_key_iv: "f2494fc9e8b2588023133a7d981d0d56"
server_key_version: "encryptionkey@0"
I20260811 06:29:23.314790 30709 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:29:23.337764 30709 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:29:23.342386 30709 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:29:23.342774 30709 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:29:23.343890 30709 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:29:23.345467 30709 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:29:23.345618 30709 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260811 06:29:23.345786 30709 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:29:23.345871 30709 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:29:23.384805 30709 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.193:40713
I20260811 06:29:23.384856 30930 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.193:40713 every 8 connection(s)
I20260811 06:29:23.386603 30709 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
I20260811 06:29:23.389154 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 30709
I20260811 06:29:23.389465 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/minicluster-data/ts-0/wal/instance
I20260811 06:29:23.390456 29631 external_mini_cluster.cc:1428] Setting key 19a305e5f13da8287f344fb9fd14f8aa
Aug 11 06:29:23 dist-test-slave-gkw7 krb5kdc[30534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429763, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.193@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.28.239.194@KRBTEST.COM; defaulting to no policy
I20260811 06:29:23.415345 30936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:23.390185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.193:42769 (local address 127.28.239.254:40417)
0811 06:29:23.397953 (+ 7768us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:23.397958 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:23.397976 (+ 18us) server_negotiation.cc:407] Connection header received
0811 06:29:23.398031 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:23.398035 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:23.398105 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:23.398247 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:23.400083 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:23.400911 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:23.403632 (+ 2721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:23.404087 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:23.404248 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:23.408604 (+ 4356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:29:23.408614 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:29:23.408622 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:29:23.408674 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:29:23.411792 (+ 3118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:23.412844 (+ 1052us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:23.412848 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:23.412850 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:23.412972 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:23.413511 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:23.413514 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:23.413516 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:23.413918 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:29:23.414036 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:23.414880 (+ 844us) server_negotiation.cc:299] Negotiation successful
0811 06:29:23.415078 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7647,"thread_start_us":7520,"threads_started":1}
I20260811 06:29:23.416426 30937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:23.390864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:40417 (local address 127.28.239.193:42769)
0811 06:29:23.392250 (+ 1386us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:23.392302 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 06:29:23.394392 (+ 2090us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:29:23.398542 (+ 4150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:29:23.398552 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:29:23.399220 (+ 668us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:29:23.399877 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:23.399902 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:23.401122 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:23.401127 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:23.403450 (+ 2323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:23.403465 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:23.404374 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:23.404383 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:23.404666 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:23.405450 (+ 784us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:29:23.405479 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:29:23.408315 (+ 2836us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:29:23.411965 (+ 3650us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:29:23.411976 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:29:23.411998 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:29:23.412699 (+ 701us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:29:23.413107 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:29:23.413115 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:29:23.413121 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:29:23.413392 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:29:23.414057 (+ 665us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:29:23.414074 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:29:23.414437 (+ 363us) client_negotiation.cc:769] Sending connection context
0811 06:29:23.414726 (+ 289us) client_negotiation.cc:240] Negotiation successful
0811 06:29:23.415175 (+ 449us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1052,"thread_start_us":811,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.239.194@KRBTEST.COM" created.
I20260811 06:29:23.418437 30931 heartbeater.cc:347] Connected to a master server at 127.28.239.254:40417
I20260811 06:29:23.418820 30931 heartbeater.cc:464] Registering TS with master...
I20260811 06:29:23.419698 30931 heartbeater.cc:511] Master 127.28.239.254:40417 requested a full tablet report, sending...
I20260811 06:29:23.422024 30596 ts_manager.cc:194] Registered new tserver with Master: 53f641564e96438bb6170e32d7bc91d6 (127.28.239.193:40713)
I20260811 06:29:23.423538 30596 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.239.193@KRBTEST.COM'} at 127.28.239.193:42769
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.239.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.194.keytab.
Entry for principal kudu/127.28.239.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.194.keytab.
I20260811 06:29:23.441517 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.194:0
--local_ip_for_outbound_sockets=127.28.239.194
--webserver_interface=127.28.239.194
--webserver_port=0
--tserver_master_addrs=127.28.239.254:40417
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.194.keytab
--principal=kudu/127.28.239.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:43819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:29:23.779417 30940 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:29:23.779723 30940 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:29:23.779909 30940 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:29:23.787429 30940 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:29:23.787554 30940 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:29:23.787730 30940 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.194
I20260811 06:29:23.798026 30940 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:43819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.194.keytab
--principal=kudu/127.28.239.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.239.194
--webserver_port=0
--tserver_master_addrs=127.28.239.254:40417
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.194
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:29:23.799422 30940 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:29:23.800864 30940 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:29:23.814720 30948 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:29:23.814878 30951 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:29:23.814754 30947 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:29:23.816039 30940 server_base.cc:1034] running on GCE node
I20260811 06:29:23.816694 30940 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:29:23.817713 30940 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:29:23.818951 30940 hybrid_clock.cc:648] HybridClock initialized: now 1786429763818913 us; error 29 us; skew 500 ppm
I20260811 06:29:23.819264 30940 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:29:23 dist-test-slave-gkw7 krb5kdc[30534](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429763, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:29:23.822889 30940 init.cc:377] Logged in from keytab as kudu/127.28.239.194@KRBTEST.COM (short username kudu)
I20260811 06:29:23.824311 30940 webserver.cc:460] Webserver started at http://127.28.239.194:46841/ using document root <none> and password file <none>
I20260811 06:29:23.824985 30940 fs_manager.cc:359] Metadata directory not provided
I20260811 06:29:23.825085 30940 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:29:23.825346 30940 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:29:23.828343 30940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/minicluster-data/ts-1/data/instance:
uuid: "f76578e8d7614635b90c8919b9ec5d67"
format_stamp: "Formatted at 2026-08-11 06:29:23 on dist-test-slave-gkw7"
server_key: "34de127d574bbc7ea68b6582d43a7dbc"
server_key_iv: "519774f6f27d5c63a2f9027ab20d7910"
server_key_version: "encryptionkey@0"
I20260811 06:29:23.829093 30940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/minicluster-data/ts-1/wal/instance:
uuid: "f76578e8d7614635b90c8919b9ec5d67"
format_stamp: "Formatted at 2026-08-11 06:29:23 on dist-test-slave-gkw7"
server_key: "34de127d574bbc7ea68b6582d43a7dbc"
server_key_iv: "519774f6f27d5c63a2f9027ab20d7910"
server_key_version: "encryptionkey@0"
I20260811 06:29:23.834403 30940 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 06:29:23.837985 30963 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:29:23.839416 30940 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260811 06:29:23.839594 30940 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/minicluster-data/ts-1/wal
uuid: "f76578e8d7614635b90c8919b9ec5d67"
format_stamp: "Formatted at 2026-08-11 06:29:23 on dist-test-slave-gkw7"
server_key: "34de127d574bbc7ea68b6582d43a7dbc"
server_key_iv: "519774f6f27d5c63a2f9027ab20d7910"
server_key_version: "encryptionkey@0"
I20260811 06:29:23.839816 30940 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:29:23.855680 30940 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:29:23.860378 30940 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:29:23.860710 30940 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:29:23.861950 30940 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:29:23.863386 30940 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:29:23.863477 30940 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:29:23.863631 30940 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:29:23.863690 30940 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:29:23.901630 30940 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.194:33805
I20260811 06:29:23.901746 31161 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.194:33805 every 8 connection(s)
I20260811 06:29:23.903172 30940 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
I20260811 06:29:23.912194 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 30940
I20260811 06:29:23.912484 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/minicluster-data/ts-1/wal/instance
I20260811 06:29:23.913071 29631 external_mini_cluster.cc:1428] Setting key 1ef438577d6196548ca14fa8fe105796
Aug 11 06:29:23 dist-test-slave-gkw7 krb5kdc[30534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429763, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.194@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:29:23.926597 30936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:23.906054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.194:43535 (local address 127.28.239.254:40417)
0811 06:29:23.906277 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:23.906284 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:23.907611 (+ 1327us) server_negotiation.cc:407] Connection header received
0811 06:29:23.909485 (+ 1874us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:23.909490 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:23.909634 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:23.909886 (+ 252us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:23.911362 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:23.912162 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:23.914513 (+ 2351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:23.915247 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:23.915391 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:23.920035 (+ 4644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:29:23.920041 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:29:23.920044 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:29:23.920078 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:29:23.922934 (+ 2856us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:23.924003 (+ 1069us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:23.924007 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:23.924010 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:23.924158 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:23.924685 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:23.924688 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:23.924690 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:23.924969 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:29:23.925086 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:23.926124 (+ 1038us) server_negotiation.cc:299] Negotiation successful
0811 06:29:23.926329 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 06:29:23.928359 31164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:23.906659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:40417 (local address 127.28.239.194:43535)
0811 06:29:23.907422 (+ 763us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:23.907471 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 06:29:23.909185 (+ 1714us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:29:23.909979 (+ 794us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:29:23.909987 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:29:23.910571 (+ 584us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:29:23.911161 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:23.911179 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:23.912376 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:23.912382 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:23.914370 (+ 1988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:23.914382 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:23.915526 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:23.915534 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:23.915867 (+ 333us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:23.916611 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:29:23.916647 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:29:23.919767 (+ 3120us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:29:23.923117 (+ 3350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:29:23.923131 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:29:23.923153 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:29:23.923867 (+ 714us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:29:23.924278 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:29:23.924286 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:29:23.924292 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:29:23.924571 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:29:23.925107 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:29:23.925119 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:29:23.925698 (+ 579us) client_negotiation.cc:769] Sending connection context
0811 06:29:23.926032 (+ 334us) client_negotiation.cc:240] Negotiation successful
0811 06:29:23.926715 (+ 683us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":305,"threads_started":1}
I20260811 06:29:23.930028 31162 heartbeater.cc:347] Connected to a master server at 127.28.239.254:40417
I20260811 06:29:23.930377 31162 heartbeater.cc:464] Registering TS with master...
I20260811 06:29:23.931277 31162 heartbeater.cc:511] Master 127.28.239.254:40417 requested a full tablet report, sending...
I20260811 06:29:23.932935 30596 ts_manager.cc:194] Registered new tserver with Master: f76578e8d7614635b90c8919b9ec5d67 (127.28.239.194:33805)
I20260811 06:29:23.933858 30596 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.239.194@KRBTEST.COM'} at 127.28.239.194:43535
WARNING: no policy specified for kudu/127.28.239.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.239.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.239.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.195.keytab.
Entry for principal kudu/127.28.239.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.195.keytab.
I20260811 06:29:23.965267 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.195:0
--local_ip_for_outbound_sockets=127.28.239.195
--webserver_interface=127.28.239.195
--webserver_port=0
--tserver_master_addrs=127.28.239.254:40417
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.195.keytab
--principal=kudu/127.28.239.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:43819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:29:24.308298 31168 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:29:24.308645 31168 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:29:24.308840 31168 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:29:24.316346 31168 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:29:24.316516 31168 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:29:24.316720 31168 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.195
I20260811 06:29:24.326890 31168 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:43819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.195.keytab
--principal=kudu/127.28.239.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.239.195
--webserver_port=0
--tserver_master_addrs=127.28.239.254:40417
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.195
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:29:24.328357 31168 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:29:24.329778 31168 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:29:24.343436 31178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:29:24.343439 31176 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:29:24.343438 31175 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:29:24.344827 31168 server_base.cc:1034] running on GCE node
I20260811 06:29:24.345477 31168 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:29:24.346508 31168 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:29:24.347705 31168 hybrid_clock.cc:648] HybridClock initialized: now 1786429764347634 us; error 65 us; skew 500 ppm
I20260811 06:29:24.348028 31168 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:29:24 dist-test-slave-gkw7 krb5kdc[30534](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429764, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:29:24.351486 31168 init.cc:377] Logged in from keytab as kudu/127.28.239.195@KRBTEST.COM (short username kudu)
I20260811 06:29:24.353031 31168 webserver.cc:460] Webserver started at http://127.28.239.195:42343/ using document root <none> and password file <none>
I20260811 06:29:24.353796 31168 fs_manager.cc:359] Metadata directory not provided
I20260811 06:29:24.353904 31168 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:29:24.354251 31168 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:29:24.357266 31168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/minicluster-data/ts-2/data/instance:
uuid: "b25b29c842184761b69f2b47f17c6fdc"
format_stamp: "Formatted at 2026-08-11 06:29:24 on dist-test-slave-gkw7"
server_key: "cd755cbe42e05e910d56a2fd64316d72"
server_key_iv: "becb00309133585948d8954a7de7965d"
server_key_version: "encryptionkey@0"
I20260811 06:29:24.358242 31168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/minicluster-data/ts-2/wal/instance:
uuid: "b25b29c842184761b69f2b47f17c6fdc"
format_stamp: "Formatted at 2026-08-11 06:29:24 on dist-test-slave-gkw7"
server_key: "cd755cbe42e05e910d56a2fd64316d72"
server_key_iv: "becb00309133585948d8954a7de7965d"
server_key_version: "encryptionkey@0"
I20260811 06:29:24.363675 31168 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 06:29:24.367187 31190 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:29:24.368495 31168 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 06:29:24.368678 31168 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/minicluster-data/ts-2/wal
uuid: "b25b29c842184761b69f2b47f17c6fdc"
format_stamp: "Formatted at 2026-08-11 06:29:24 on dist-test-slave-gkw7"
server_key: "cd755cbe42e05e910d56a2fd64316d72"
server_key_iv: "becb00309133585948d8954a7de7965d"
server_key_version: "encryptionkey@0"
I20260811 06:29:24.368899 31168 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:29:24.380947 31168 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:29:24.386143 31168 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:29:24.386471 31168 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:29:24.387617 31168 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:29:24.389323 31168 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:29:24.389413 31168 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:29:24.389562 31168 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:29:24.389631 31168 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260811 06:29:24.422984 31168 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.195:40403
I20260811 06:29:24.423019 31370 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.195:40403 every 8 connection(s)
I20260811 06:29:24.424335 31168 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
I20260811 06:29:24.425714 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 31168
I20260811 06:29:24.425932 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786429755927508-29631-0/minicluster-data/ts-2/wal/instance
I20260811 06:29:24.426487 29631 external_mini_cluster.cc:1428] Setting key e75f769468ca74bb277c88d74e1b4758
I20260811 06:29:24.426762 30931 heartbeater.cc:503] Master 127.28.239.254:40417 was elected leader, sending a full tablet report...
Aug 11 06:29:24 dist-test-slave-gkw7 krb5kdc[30534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429764, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.195@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:29:24.446691 31375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:24.428931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.195:48337 (local address 127.28.239.254:40417)
0811 06:29:24.429996 (+ 1065us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:24.430003 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:24.430024 (+ 21us) server_negotiation.cc:407] Connection header received
0811 06:29:24.431551 (+ 1527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:24.431557 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:24.431639 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:24.431778 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:24.433297 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:24.434183 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:24.435747 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:24.436169 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:24.436314 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:24.439926 (+ 3612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:29:24.439931 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:29:24.439934 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:29:24.439965 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:29:24.443508 (+ 3543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:24.444221 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:24.444228 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:24.444232 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:24.444381 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:24.444828 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:24.444835 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:24.444839 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:24.445119 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:29:24.445277 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:24.446051 (+ 774us) server_negotiation.cc:299] Negotiation successful
0811 06:29:24.446277 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":944,"thread_start_us":814,"threads_started":1}
I20260811 06:29:24.447700 31374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:24.428324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:40417 (local address 127.28.239.195:48337)
0811 06:29:24.429210 (+ 886us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:24.429271 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 06:29:24.431284 (+ 2013us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:29:24.432023 (+ 739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:29:24.432037 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:29:24.432576 (+ 539us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:29:24.433131 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:24.433147 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:24.434356 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:24.434361 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:24.435594 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:24.435611 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:24.436411 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:24.436418 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:24.436649 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:24.437340 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:29:24.437363 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:29:24.439706 (+ 2343us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:29:24.443642 (+ 3936us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:29:24.443651 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:29:24.443669 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:29:24.444099 (+ 430us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:29:24.444503 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:29:24.444509 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:29:24.444512 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:29:24.444704 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:29:24.445252 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:29:24.445271 (+ 19us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:29:24.445673 (+ 402us) client_negotiation.cc:769] Sending connection context
0811 06:29:24.445957 (+ 284us) client_negotiation.cc:240] Negotiation successful
0811 06:29:24.446514 (+ 557us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":583,"thread_start_us":342,"threads_started":1}
I20260811 06:29:24.449242 31371 heartbeater.cc:347] Connected to a master server at 127.28.239.254:40417
I20260811 06:29:24.449579 31371 heartbeater.cc:464] Registering TS with master...
I20260811 06:29:24.450305 31371 heartbeater.cc:511] Master 127.28.239.254:40417 requested a full tablet report, sending...
I20260811 06:29:24.451797 30596 ts_manager.cc:194] Registered new tserver with Master: b25b29c842184761b69f2b47f17c6fdc (127.28.239.195:40403)
I20260811 06:29:24.452728 30596 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.239.195@KRBTEST.COM'} at 127.28.239.195:48337
I20260811 06:29:24.455662 29631 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 06:29:24.476418 31375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:24.463002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56468 (local address 127.28.239.254:40417)
0811 06:29:24.463195 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:24.463203 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:24.463640 (+ 437us) server_negotiation.cc:407] Connection header received
0811 06:29:24.463858 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:24.463863 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:24.463970 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:24.464078 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:24.465641 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:24.466469 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:24.467799 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:24.468231 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:24.468384 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:24.470773 (+ 2389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:29:24.470779 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:29:24.470782 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:29:24.470815 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:29:24.473887 (+ 3072us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:24.474556 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:24.474560 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:24.474563 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:24.474679 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:24.475012 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:24.475016 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:24.475018 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:24.475332 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:29:24.475516 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:24.475835 (+ 319us) server_negotiation.cc:299] Negotiation successful
0811 06:29:24.476003 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 06:29:24.484938 30596 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56468:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:29:24.493289 30596 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:29:24.531298 31392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:24.510239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56640 (local address 127.28.239.193:40713)
0811 06:29:24.523306 (+ 13067us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:24.523312 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:24.523338 (+ 26us) server_negotiation.cc:407] Connection header received
0811 06:29:24.523419 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:24.523425 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:24.523631 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:24.523844 (+ 213us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:29:24.524661 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:24.526343 (+ 1682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:24.528888 (+ 2545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:24.529999 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:24.530245 (+ 246us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:24.530456 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:24.530767 (+ 311us) server_negotiation.cc:299] Negotiation successful
0811 06:29:24.530941 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12904,"thread_start_us":12762,"threads_started":1}
I20260811 06:29:24.531389 31395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:24.510228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.193:40713 (local address 127.0.0.1:56640)
0811 06:29:24.522546 (+ 12318us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:24.522610 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0811 06:29:24.522952 (+ 342us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:29:24.524099 (+ 1147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:29:24.524107 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:29:24.524143 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:29:24.524501 (+ 358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:24.524511 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:24.526525 (+ 2014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:24.526536 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:24.528732 (+ 2196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:24.528744 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:24.530241 (+ 1497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:24.530251 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:24.530470 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:24.530539 (+ 69us) client_negotiation.cc:769] Sending connection context
0811 06:29:24.530708 (+ 169us) client_negotiation.cc:240] Negotiation successful
0811 06:29:24.530926 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11955,"mutex_wait_us":52,"thread_start_us":11754,"threads_started":1}
I20260811 06:29:24.533977 31398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:24.510314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.194:33805 (local address 127.0.0.1:35844)
0811 06:29:24.523737 (+ 13423us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:24.523757 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 06:29:24.523888 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:29:24.524981 (+ 1093us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:29:24.524986 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:29:24.525012 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:29:24.526006 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:24.526025 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:24.528045 (+ 2020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:24.528051 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:24.530433 (+ 2382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:24.530447 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:24.532991 (+ 2544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:24.532999 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:24.533201 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:24.533245 (+ 44us) client_negotiation.cc:769] Sending connection context
0811 06:29:24.533362 (+ 117us) client_negotiation.cc:240] Negotiation successful
0811 06:29:24.533575 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13238,"mutex_wait_us":5,"thread_start_us":13054,"threads_started":1}
I20260811 06:29:24.534134 31400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:24.510578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35844 (local address 127.28.239.194:33805)
0811 06:29:24.521460 (+ 10882us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:24.521476 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:24.524033 (+ 2557us) server_negotiation.cc:407] Connection header received
0811 06:29:24.524185 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:24.524194 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:24.524458 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:24.524663 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:29:24.526340 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:24.527901 (+ 1561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:24.530601 (+ 2700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:24.532770 (+ 2169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:24.532955 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:24.533135 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:24.533465 (+ 330us) server_negotiation.cc:299] Negotiation successful
0811 06:29:24.533723 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10624,"thread_start_us":10937,"threads_started":1}
I20260811 06:29:24.535712 31405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:24.523895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.195:40403 (local address 127.0.0.1:45002)
0811 06:29:24.526681 (+ 2786us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:24.526704 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 06:29:24.526841 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:29:24.527447 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:29:24.527451 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:29:24.527477 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:29:24.527907 (+ 430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:24.527919 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:24.530025 (+ 2106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:24.530031 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:24.532298 (+ 2267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:24.532310 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:24.534808 (+ 2498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:24.534816 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:24.535010 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:24.535047 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 06:29:24.535186 (+ 139us) client_negotiation.cc:240] Negotiation successful
0811 06:29:24.535330 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2643,"thread_start_us":2504,"threads_started":1}
I20260811 06:29:24.535810 31402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:24.524303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45002 (local address 127.28.239.195:40403)
0811 06:29:24.525217 (+ 914us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:24.525225 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:24.526818 (+ 1593us) server_negotiation.cc:407] Connection header received
0811 06:29:24.526998 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:24.527005 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:24.527231 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:24.527437 (+ 206us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:29:24.528137 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:24.529854 (+ 1717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:24.532467 (+ 2613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:24.534543 (+ 2076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:24.534808 (+ 265us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:24.534986 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:24.535234 (+ 248us) server_negotiation.cc:299] Negotiation successful
0811 06:29:24.535406 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":701,"thread_start_us":533,"threads_started":1}
I20260811 06:29:24.537096 30824 tablet_service.cc:1467] Processing CreateTablet for tablet d008734c3bd6470d8ea95b097abdf3ab (DEFAULT_TABLE table=test-table [id=b6cfc34c5b3a4deea72d5e5db147ba68]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:29:24.538260 31048 tablet_service.cc:1467] Processing CreateTablet for tablet d008734c3bd6470d8ea95b097abdf3ab (DEFAULT_TABLE table=test-table [id=b6cfc34c5b3a4deea72d5e5db147ba68]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:29:24.539847 31048 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d008734c3bd6470d8ea95b097abdf3ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:29:24.539840 30824 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d008734c3bd6470d8ea95b097abdf3ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:29:24.541133 31278 tablet_service.cc:1467] Processing CreateTablet for tablet d008734c3bd6470d8ea95b097abdf3ab (DEFAULT_TABLE table=test-table [id=b6cfc34c5b3a4deea72d5e5db147ba68]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:29:24.542425 31278 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d008734c3bd6470d8ea95b097abdf3ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:29:24.554981 31407 tablet_bootstrap.cc:492] T d008734c3bd6470d8ea95b097abdf3ab P b25b29c842184761b69f2b47f17c6fdc: Bootstrap starting.
I20260811 06:29:24.558188 31408 tablet_bootstrap.cc:492] T d008734c3bd6470d8ea95b097abdf3ab P 53f641564e96438bb6170e32d7bc91d6: Bootstrap starting.
I20260811 06:29:24.560477 31407 tablet_bootstrap.cc:654] T d008734c3bd6470d8ea95b097abdf3ab P b25b29c842184761b69f2b47f17c6fdc: Neither blocks nor log segments found. Creating new log.
I20260811 06:29:24.561192 31409 tablet_bootstrap.cc:492] T d008734c3bd6470d8ea95b097abdf3ab P f76578e8d7614635b90c8919b9ec5d67: Bootstrap starting.
I20260811 06:29:24.562100 31407 log.cc:824] T d008734c3bd6470d8ea95b097abdf3ab P b25b29c842184761b69f2b47f17c6fdc: Log is configured to *not* fsync() on all Append() calls
I20260811 06:29:24.562815 31408 tablet_bootstrap.cc:654] T d008734c3bd6470d8ea95b097abdf3ab P 53f641564e96438bb6170e32d7bc91d6: Neither blocks nor log segments found. Creating new log.
I20260811 06:29:24.564432 31408 log.cc:824] T d008734c3bd6470d8ea95b097abdf3ab P 53f641564e96438bb6170e32d7bc91d6: Log is configured to *not* fsync() on all Append() calls
I20260811 06:29:24.564687 31409 tablet_bootstrap.cc:654] T d008734c3bd6470d8ea95b097abdf3ab P f76578e8d7614635b90c8919b9ec5d67: Neither blocks nor log segments found. Creating new log.
I20260811 06:29:24.566113 31409 log.cc:824] T d008734c3bd6470d8ea95b097abdf3ab P f76578e8d7614635b90c8919b9ec5d67: Log is configured to *not* fsync() on all Append() calls
I20260811 06:29:24.566177 31407 tablet_bootstrap.cc:492] T d008734c3bd6470d8ea95b097abdf3ab P b25b29c842184761b69f2b47f17c6fdc: No bootstrap required, opened a new log
I20260811 06:29:24.566491 31407 ts_tablet_manager.cc:1397] T d008734c3bd6470d8ea95b097abdf3ab P b25b29c842184761b69f2b47f17c6fdc: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.002s
I20260811 06:29:24.568193 31408 tablet_bootstrap.cc:492] T d008734c3bd6470d8ea95b097abdf3ab P 53f641564e96438bb6170e32d7bc91d6: No bootstrap required, opened a new log
I20260811 06:29:24.568495 31408 ts_tablet_manager.cc:1397] T d008734c3bd6470d8ea95b097abdf3ab P 53f641564e96438bb6170e32d7bc91d6: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260811 06:29:24.568912 31409 tablet_bootstrap.cc:492] T d008734c3bd6470d8ea95b097abdf3ab P f76578e8d7614635b90c8919b9ec5d67: No bootstrap required, opened a new log
I20260811 06:29:24.569164 31409 ts_tablet_manager.cc:1397] T d008734c3bd6470d8ea95b097abdf3ab P f76578e8d7614635b90c8919b9ec5d67: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260811 06:29:24.576924 31409 raft_consensus.cc:348] T d008734c3bd6470d8ea95b097abdf3ab P f76578e8d7614635b90c8919b9ec5d67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53f641564e96438bb6170e32d7bc91d6" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 40713 } } peers { permanent_uuid: "f76578e8d7614635b90c8919b9ec5d67" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 33805 } } peers { permanent_uuid: "b25b29c842184761b69f2b47f17c6fdc" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 40403 } }
I20260811 06:29:24.577325 31409 raft_consensus.cc:374] T d008734c3bd6470d8ea95b097abdf3ab P f76578e8d7614635b90c8919b9ec5d67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:29:24.577466 31409 raft_consensus.cc:729] T d008734c3bd6470d8ea95b097abdf3ab P f76578e8d7614635b90c8919b9ec5d67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f76578e8d7614635b90c8919b9ec5d67, State: Initialized, Role: FOLLOWER
I20260811 06:29:24.578207 31409 consensus_queue.cc:260] T d008734c3bd6470d8ea95b097abdf3ab P f76578e8d7614635b90c8919b9ec5d67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53f641564e96438bb6170e32d7bc91d6" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 40713 } } peers { permanent_uuid: "f76578e8d7614635b90c8919b9ec5d67" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 33805 } } peers { permanent_uuid: "b25b29c842184761b69f2b47f17c6fdc" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 40403 } }
I20260811 06:29:24.578732 31407 raft_consensus.cc:348] T d008734c3bd6470d8ea95b097abdf3ab P b25b29c842184761b69f2b47f17c6fdc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53f641564e96438bb6170e32d7bc91d6" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 40713 } } peers { permanent_uuid: "f76578e8d7614635b90c8919b9ec5d67" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 33805 } } peers { permanent_uuid: "b25b29c842184761b69f2b47f17c6fdc" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 40403 } }
I20260811 06:29:24.579281 31407 raft_consensus.cc:374] T d008734c3bd6470d8ea95b097abdf3ab P b25b29c842184761b69f2b47f17c6fdc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:29:24.579424 31407 raft_consensus.cc:729] T d008734c3bd6470d8ea95b097abdf3ab P b25b29c842184761b69f2b47f17c6fdc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b25b29c842184761b69f2b47f17c6fdc, State: Initialized, Role: FOLLOWER
I20260811 06:29:24.579500 31162 heartbeater.cc:503] Master 127.28.239.254:40417 was elected leader, sending a full tablet report...
I20260811 06:29:24.580098 31407 consensus_queue.cc:260] T d008734c3bd6470d8ea95b097abdf3ab P b25b29c842184761b69f2b47f17c6fdc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53f641564e96438bb6170e32d7bc91d6" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 40713 } } peers { permanent_uuid: "f76578e8d7614635b90c8919b9ec5d67" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 33805 } } peers { permanent_uuid: "b25b29c842184761b69f2b47f17c6fdc" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 40403 } }
I20260811 06:29:24.580957 31408 raft_consensus.cc:348] T d008734c3bd6470d8ea95b097abdf3ab P 53f641564e96438bb6170e32d7bc91d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53f641564e96438bb6170e32d7bc91d6" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 40713 } } peers { permanent_uuid: "f76578e8d7614635b90c8919b9ec5d67" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 33805 } } peers { permanent_uuid: "b25b29c842184761b69f2b47f17c6fdc" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 40403 } }
I20260811 06:29:24.581501 31371 heartbeater.cc:503] Master 127.28.239.254:40417 was elected leader, sending a full tablet report...
I20260811 06:29:24.581507 31408 raft_consensus.cc:374] T d008734c3bd6470d8ea95b097abdf3ab P 53f641564e96438bb6170e32d7bc91d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:29:24.581643 31408 raft_consensus.cc:729] T d008734c3bd6470d8ea95b097abdf3ab P 53f641564e96438bb6170e32d7bc91d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53f641564e96438bb6170e32d7bc91d6, State: Initialized, Role: FOLLOWER
I20260811 06:29:24.582229 31407 ts_tablet_manager.cc:1428] T d008734c3bd6470d8ea95b097abdf3ab P b25b29c842184761b69f2b47f17c6fdc: Time spent starting tablet: real 0.016s user 0.009s sys 0.007s
I20260811 06:29:24.582427 31408 consensus_queue.cc:260] T d008734c3bd6470d8ea95b097abdf3ab P 53f641564e96438bb6170e32d7bc91d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53f641564e96438bb6170e32d7bc91d6" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 40713 } } peers { permanent_uuid: "f76578e8d7614635b90c8919b9ec5d67" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 33805 } } peers { permanent_uuid: "b25b29c842184761b69f2b47f17c6fdc" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 40403 } }
I20260811 06:29:24.583555 31409 ts_tablet_manager.cc:1428] T d008734c3bd6470d8ea95b097abdf3ab P f76578e8d7614635b90c8919b9ec5d67: Time spent starting tablet: real 0.014s user 0.008s sys 0.004s
I20260811 06:29:24.585399 31408 ts_tablet_manager.cc:1428] T d008734c3bd6470d8ea95b097abdf3ab P 53f641564e96438bb6170e32d7bc91d6: Time spent starting tablet: real 0.017s user 0.010s sys 0.009s
W20260811 06:29:24.639364 30932 tablet.cc:2369] T d008734c3bd6470d8ea95b097abdf3ab P 53f641564e96438bb6170e32d7bc91d6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 06:29:24.655531 31163 tablet.cc:2369] T d008734c3bd6470d8ea95b097abdf3ab P f76578e8d7614635b90c8919b9ec5d67: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 06:29:24.676159 31373 tablet.cc:2369] T d008734c3bd6470d8ea95b097abdf3ab P b25b29c842184761b69f2b47f17c6fdc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:29:24.880198 31418 raft_consensus.cc:482] T d008734c3bd6470d8ea95b097abdf3ab P b25b29c842184761b69f2b47f17c6fdc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:29:24.880509 31418 raft_consensus.cc:504] T d008734c3bd6470d8ea95b097abdf3ab P b25b29c842184761b69f2b47f17c6fdc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53f641564e96438bb6170e32d7bc91d6" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 40713 } } peers { permanent_uuid: "f76578e8d7614635b90c8919b9ec5d67" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 33805 } } peers { permanent_uuid: "b25b29c842184761b69f2b47f17c6fdc" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 40403 } }
I20260811 06:29:24.882726 31418 leader_election.cc:288] T d008734c3bd6470d8ea95b097abdf3ab P b25b29c842184761b69f2b47f17c6fdc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 53f641564e96438bb6170e32d7bc91d6 (127.28.239.193:40713), f76578e8d7614635b90c8919b9ec5d67 (127.28.239.194:33805)
I20260811 06:29:24.888937 31374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:24.883050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.194:33805 (local address 127.28.239.195:40579)
0811 06:29:24.883293 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:24.883318 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 06:29:24.883466 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:29:24.883945 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:29:24.883950 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:29:24.883981 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:29:24.884287 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:24.884301 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:24.885586 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:24.885592 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:24.887021 (+ 1429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:24.887034 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:24.888193 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:24.888199 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:24.888379 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:24.888402 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 06:29:24.888504 (+ 102us) client_negotiation.cc:240] Negotiation successful
0811 06:29:24.888641 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84}
I20260811 06:29:24.888934 31400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:24.883332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.195:40579 (local address 127.28.239.194:33805)
0811 06:29:24.883581 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:24.883587 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:24.883610 (+ 23us) server_negotiation.cc:407] Connection header received
0811 06:29:24.883705 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:24.883713 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:24.883795 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:24.883921 (+ 126us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:29:24.884451 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:24.885442 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:24.887210 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:24.888027 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:24.888192 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:24.888269 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:24.888542 (+ 273us) server_negotiation.cc:299] Negotiation successful
0811 06:29:24.888683 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260811 06:29:24.890086 31420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:24.883281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.193:40713 (local address 127.28.239.195:39337)
0811 06:29:24.884066 (+ 785us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:24.884084 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 06:29:24.884251 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:29:24.884652 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:29:24.884657 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:29:24.884684 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:29:24.885083 (+ 399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:24.885094 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:24.886504 (+ 1410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:24.886519 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:24.887925 (+ 1406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:24.887938 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:24.889350 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:24.889358 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:24.889567 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:24.889599 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 06:29:24.889721 (+ 122us) client_negotiation.cc:240] Negotiation successful
0811 06:29:24.889848 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":655,"thread_start_us":499,"threads_started":1}
I20260811 06:29:24.890236 31392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:24.883478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.195:39337 (local address 127.28.239.193:40713)
0811 06:29:24.883687 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:24.883695 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:24.884212 (+ 517us) server_negotiation.cc:407] Connection header received
0811 06:29:24.884416 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:24.884422 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:24.884512 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:24.884624 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:29:24.885274 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:24.886321 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:24.888114 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:24.889023 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:24.889230 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:24.889305 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:24.889773 (+ 468us) server_negotiation.cc:299] Negotiation successful
0811 06:29:24.889956 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 06:29:24.890609 31084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d008734c3bd6470d8ea95b097abdf3ab" candidate_uuid: "b25b29c842184761b69f2b47f17c6fdc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f76578e8d7614635b90c8919b9ec5d67" is_pre_election: true
I20260811 06:29:24.891593 31084 raft_consensus.cc:2454] T d008734c3bd6470d8ea95b097abdf3ab P f76578e8d7614635b90c8919b9ec5d67 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b25b29c842184761b69f2b47f17c6fdc in term 0.
I20260811 06:29:24.891526 30857 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d008734c3bd6470d8ea95b097abdf3ab" candidate_uuid: "b25b29c842184761b69f2b47f17c6fdc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53f641564e96438bb6170e32d7bc91d6" is_pre_election: true
I20260811 06:29:24.892050 30857 raft_consensus.cc:2454] T d008734c3bd6470d8ea95b097abdf3ab P 53f641564e96438bb6170e32d7bc91d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b25b29c842184761b69f2b47f17c6fdc in term 0.
I20260811 06:29:24.892608 31206 leader_election.cc:302] T d008734c3bd6470d8ea95b097abdf3ab P b25b29c842184761b69f2b47f17c6fdc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b25b29c842184761b69f2b47f17c6fdc, f76578e8d7614635b90c8919b9ec5d67; no voters:
I20260811 06:29:24.893095 31418 raft_consensus.cc:2781] T d008734c3bd6470d8ea95b097abdf3ab P b25b29c842184761b69f2b47f17c6fdc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:29:24.893251 31418 raft_consensus.cc:482] T d008734c3bd6470d8ea95b097abdf3ab P b25b29c842184761b69f2b47f17c6fdc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:29:24.893379 31418 raft_consensus.cc:3037] T d008734c3bd6470d8ea95b097abdf3ab P b25b29c842184761b69f2b47f17c6fdc [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:29:24.895731 31418 raft_consensus.cc:504] T d008734c3bd6470d8ea95b097abdf3ab P b25b29c842184761b69f2b47f17c6fdc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53f641564e96438bb6170e32d7bc91d6" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 40713 } } peers { permanent_uuid: "f76578e8d7614635b90c8919b9ec5d67" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 33805 } } peers { permanent_uuid: "b25b29c842184761b69f2b47f17c6fdc" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 40403 } }
I20260811 06:29:24.896386 31418 leader_election.cc:288] T d008734c3bd6470d8ea95b097abdf3ab P b25b29c842184761b69f2b47f17c6fdc [CANDIDATE]: Term 1 election: Requested vote from peers 53f641564e96438bb6170e32d7bc91d6 (127.28.239.193:40713), f76578e8d7614635b90c8919b9ec5d67 (127.28.239.194:33805)
I20260811 06:29:24.896950 31084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d008734c3bd6470d8ea95b097abdf3ab" candidate_uuid: "b25b29c842184761b69f2b47f17c6fdc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f76578e8d7614635b90c8919b9ec5d67"
I20260811 06:29:24.896997 30857 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d008734c3bd6470d8ea95b097abdf3ab" candidate_uuid: "b25b29c842184761b69f2b47f17c6fdc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53f641564e96438bb6170e32d7bc91d6"
I20260811 06:29:24.897167 31084 raft_consensus.cc:3037] T d008734c3bd6470d8ea95b097abdf3ab P f76578e8d7614635b90c8919b9ec5d67 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:29:24.897203 30857 raft_consensus.cc:3037] T d008734c3bd6470d8ea95b097abdf3ab P 53f641564e96438bb6170e32d7bc91d6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:29:24.899468 31084 raft_consensus.cc:2454] T d008734c3bd6470d8ea95b097abdf3ab P f76578e8d7614635b90c8919b9ec5d67 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b25b29c842184761b69f2b47f17c6fdc in term 1.
I20260811 06:29:24.899627 30857 raft_consensus.cc:2454] T d008734c3bd6470d8ea95b097abdf3ab P 53f641564e96438bb6170e32d7bc91d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b25b29c842184761b69f2b47f17c6fdc in term 1.
I20260811 06:29:24.899967 31206 leader_election.cc:302] T d008734c3bd6470d8ea95b097abdf3ab P b25b29c842184761b69f2b47f17c6fdc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b25b29c842184761b69f2b47f17c6fdc, f76578e8d7614635b90c8919b9ec5d67; no voters:
I20260811 06:29:24.900256 31418 raft_consensus.cc:2781] T d008734c3bd6470d8ea95b097abdf3ab P b25b29c842184761b69f2b47f17c6fdc [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:29:24.900782 31418 raft_consensus.cc:686] T d008734c3bd6470d8ea95b097abdf3ab P b25b29c842184761b69f2b47f17c6fdc [term 1 LEADER]: Becoming Leader. State: Replica: b25b29c842184761b69f2b47f17c6fdc, State: Running, Role: LEADER
I20260811 06:29:24.901347 31418 consensus_queue.cc:237] T d008734c3bd6470d8ea95b097abdf3ab P b25b29c842184761b69f2b47f17c6fdc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53f641564e96438bb6170e32d7bc91d6" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 40713 } } peers { permanent_uuid: "f76578e8d7614635b90c8919b9ec5d67" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 33805 } } peers { permanent_uuid: "b25b29c842184761b69f2b47f17c6fdc" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 40403 } }
I20260811 06:29:24.906904 30596 catalog_manager.cc:5495] T d008734c3bd6470d8ea95b097abdf3ab P b25b29c842184761b69f2b47f17c6fdc reported cstate change: term changed from 0 to 1, leader changed from <none> to b25b29c842184761b69f2b47f17c6fdc (127.28.239.195). New cstate: current_term: 1 leader_uuid: "b25b29c842184761b69f2b47f17c6fdc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53f641564e96438bb6170e32d7bc91d6" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 40713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f76578e8d7614635b90c8919b9ec5d67" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 33805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b25b29c842184761b69f2b47f17c6fdc" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 40403 } health_report { overall_health: HEALTHY } } }
Aug 11 06:29:24 dist-test-slave-gkw7 krb5kdc[30534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429762, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.239.193@KRBTEST.COM
I20260811 06:29:24.982030 31392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:24.965701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56656 (local address 127.28.239.193:40713)
0811 06:29:24.965956 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:24.965966 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:24.966418 (+ 452us) server_negotiation.cc:407] Connection header received
0811 06:29:24.966634 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:24.966646 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:24.966726 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:24.966839 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:24.968499 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:24.969271 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:24.970666 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:24.971115 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:24.971252 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:24.975696 (+ 4444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:29:24.975714 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:29:24.975732 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:29:24.975769 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:29:24.978988 (+ 3219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:24.979687 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:24.979695 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:24.979698 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:24.979843 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:24.980272 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:24.980276 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:24.980278 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:24.980726 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:29:24.981045 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:24.981390 (+ 345us) server_negotiation.cc:299] Negotiation successful
0811 06:29:24.981668 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
Aug 11 06:29:25 dist-test-slave-gkw7 krb5kdc[30534](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429765, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 06:29:25 dist-test-slave-gkw7 krb5kdc[30534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429765, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.28.239.193@KRBTEST.COM
I20260811 06:29:25.037223 31392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:25.021874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56658 (local address 127.28.239.193:40713)
0811 06:29:25.022104 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:25.022113 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:25.022496 (+ 383us) server_negotiation.cc:407] Connection header received
0811 06:29:25.022720 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:25.022725 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:25.022801 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:25.022913 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:25.024318 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:25.025089 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:25.026421 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:25.026872 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:25.027017 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:25.031329 (+ 4312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:29:25.031335 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:29:25.031339 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:29:25.031381 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:29:25.034493 (+ 3112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:25.035171 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:25.035176 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:25.035178 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:25.035317 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:25.035665 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:25.035669 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:25.035671 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:25.035956 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:29:25.036245 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:25.036617 (+ 372us) server_negotiation.cc:299] Negotiation successful
0811 06:29:25.036821 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
Aug 11 06:29:25 dist-test-slave-gkw7 krb5kdc[30534](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429765, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 06:29:25 dist-test-slave-gkw7 krb5kdc[30534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429765, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.239.193@KRBTEST.COM
I20260811 06:29:25.105298 31392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:25.091038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56672 (local address 127.28.239.193:40713)
0811 06:29:25.091345 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:25.091355 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:25.091740 (+ 385us) server_negotiation.cc:407] Connection header received
0811 06:29:25.091908 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:25.091912 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:25.091980 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:25.092068 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:25.093465 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:25.094206 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:25.095596 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:25.096005 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:25.096101 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:25.100102 (+ 4001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:29:25.100107 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:29:25.100111 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:29:25.100144 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:29:25.102644 (+ 2500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:25.103302 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:25.103310 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:25.103318 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:25.103450 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:25.103824 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:25.103830 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:25.103833 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:25.104104 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:29:25.104389 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:25.104692 (+ 303us) server_negotiation.cc:299] Negotiation successful
0811 06:29:25.104924 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
W20260811 06:29:25.107560 30796 tablet_service.cc:2908] Rejecting scan request for tablet d008734c3bd6470d8ea95b097abdf3ab: Uninitialized: clean time has not yet been initialized
I20260811 06:29:25.112466 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 30709
I20260811 06:29:25.125958 31392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:25.113407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56678 (local address 127.28.239.193:40713)
0811 06:29:25.113741 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:25.113746 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:25.114020 (+ 274us) server_negotiation.cc:407] Connection header received
0811 06:29:25.114164 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:25.114168 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:25.114236 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:25.114332 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:25.115579 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:25.116404 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:25.117683 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:25.118198 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:25.118318 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:25.120811 (+ 2493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:29:25.120816 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:29:25.120819 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:29:25.120856 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:29:25.123543 (+ 2687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:25.124176 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:25.124180 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:25.124182 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:25.124302 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:25.124612 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:25.124615 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:25.124617 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:25.124874 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:29:25.125073 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:25.125318 (+ 245us) server_negotiation.cc:299] Negotiation successful
0811 06:29:25.125473 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260811 06:29:25.126914 30924 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:29:25.217487 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 30709
I20260811 06:29:25.244810 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 30940
Aug 11 06:29:25 dist-test-slave-gkw7 krb5kdc[30534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429765, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.239.194@KRBTEST.COM
I20260811 06:29:25.260528 31400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:25.245614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35856 (local address 127.28.239.194:33805)
0811 06:29:25.245814 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:25.245818 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:25.245889 (+ 71us) server_negotiation.cc:407] Connection header received
0811 06:29:25.246048 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:25.246052 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:25.246127 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:25.246218 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:25.247384 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:25.248135 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:25.249368 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:25.250051 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:25.250160 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:25.254238 (+ 4078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:29:25.254255 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:29:25.254274 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:29:25.254313 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:29:25.258014 (+ 3701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:25.258651 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:25.258658 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:25.258661 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:25.258805 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:25.259133 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:25.259136 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:25.259138 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:25.259519 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:29:25.259741 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:25.260090 (+ 349us) server_negotiation.cc:299] Negotiation successful
0811 06:29:25.260266 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 06:29:25.261240 31154 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:29:25.354938 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 30940
W20260811 06:29:25.382627 31206 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.28.239.194:33805 (error 108)
I20260811 06:29:25.383139 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 31168
I20260811 06:29:25.384984 31420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:25.384430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.194:33805 (local address 127.28.239.195:39147)
0811 06:29:25.384644 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:25.384836 (+ 192us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.194:33805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260811 06:29:25.385299 31206 consensus_peers.cc:487] T d008734c3bd6470d8ea95b097abdf3ab P b25b29c842184761b69f2b47f17c6fdc -> Peer f76578e8d7614635b90c8919b9ec5d67 (127.28.239.194:33805): Couldn't send request to peer f76578e8d7614635b90c8919b9ec5d67. Status: Network error: Client connection negotiation failed: client connection to 127.28.239.194:33805: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
Aug 11 06:29:25 dist-test-slave-gkw7 krb5kdc[30534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429765, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.239.195@KRBTEST.COM
I20260811 06:29:25.399718 31449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:25.383882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45016 (local address 127.28.239.195:40403)
0811 06:29:25.384409 (+ 527us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:25.384421 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:25.384448 (+ 27us) server_negotiation.cc:407] Connection header received
0811 06:29:25.384515 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:25.384518 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:25.384624 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:25.384799 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:25.386622 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:25.387333 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:25.388659 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:25.389106 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:25.389246 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:25.393499 (+ 4253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:29:25.393511 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:29:25.393570 (+ 59us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:29:25.393618 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:29:25.396665 (+ 3047us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:25.397397 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:25.397406 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:25.397410 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:25.397636 (+ 226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:25.398042 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:25.398047 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:25.398051 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:25.398458 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:29:25.398779 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:25.399083 (+ 304us) server_negotiation.cc:299] Negotiation successful
0811 06:29:25.399345 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":210,"threads_started":1}
I20260811 06:29:25.400542 31365 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:29:25.491295 31420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:25.490807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.193:40713 (local address 127.28.239.195:51451)
0811 06:29:25.490990 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:25.491119 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.193:40713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260811 06:29:25.492506 31420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:25.492050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.193:40713 (local address 127.28.239.195:48655)
0811 06:29:25.492262 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:25.492339 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.193:40713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120}
W20260811 06:29:25.492820 31208 consensus_peers.cc:487] T d008734c3bd6470d8ea95b097abdf3ab P b25b29c842184761b69f2b47f17c6fdc -> Peer 53f641564e96438bb6170e32d7bc91d6 (127.28.239.193:40713): Couldn't send request to peer 53f641564e96438bb6170e32d7bc91d6. Status: Network error: Client connection negotiation failed: client connection to 127.28.239.193:40713: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 06:29:25.498255 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 31168
I20260811 06:29:25.522188 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 30549
I20260811 06:29:25.523447 30661 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:29:25.634402 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 30549
2026-08-11T06:29:25Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4914 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:29:25 dist-test-slave-gkw7 krb5kdc[31458](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:29:25 dist-test-slave-gkw7 krb5kdc[31458](info): set up 2 sockets
Aug 11 06:29:25 dist-test-slave-gkw7 krb5kdc[31458](info): commencing operation
krb5kdc: starting...
W20260811 06:29:26.965440 29631 mini_kdc.cc:121] Time spent starting KDC: real 1.278s user 0.003s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/krb5kdc/test-user.keytab.
Aug 11 06:29:27 dist-test-slave-gkw7 krb5kdc[31458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429767, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:29:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:29:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.239.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.239.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.239.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab.
Entry for principal kudu/127.28.239.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab.
I20260811 06:29:27.182327 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:43627
--webserver_interface=127.28.239.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab
--principal=kudu/127.28.239.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:40295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.239.254:43627
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:29:27.527047 31476 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:29:27.527343 31476 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:29:27.527521 31476 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:29:27.534890 31476 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:29:27.534997 31476 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 06:29:27.535079 31476 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:29:27.535181 31476 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:29:27.535269 31476 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:29:27.535356 31476 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:29:27.545297 31476 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:40295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.239.254:43627
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab
--principal=kudu/127.28.239.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:43627
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.239.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:29:27.546667 31476 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:29:27.548030 31476 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:29:27.560037 31482 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:29:27.560047 31485 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:29:27.560070 31483 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:29:27.560995 31476 server_base.cc:1034] running on GCE node
I20260811 06:29:27.561936 31476 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:29:27.563386 31476 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:29:27.564666 31476 hybrid_clock.cc:648] HybridClock initialized: now 1786429767564581 us; error 49 us; skew 500 ppm
I20260811 06:29:27.565032 31476 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:29:27 dist-test-slave-gkw7 krb5kdc[31458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429767, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:29:27.568455 31476 init.cc:377] Logged in from keytab as kudu/127.28.239.254@KRBTEST.COM (short username kudu)
I20260811 06:29:27.570153 31476 webserver.cc:460] Webserver started at http://127.28.239.254:34955/ using document root <none> and password file <none>
I20260811 06:29:27.570909 31476 fs_manager.cc:359] Metadata directory not provided
I20260811 06:29:27.571045 31476 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:29:27.571398 31476 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:29:27.574558 31476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/minicluster-data/master-0/data/instance:
uuid: "66e064d761374e509bbcfe0045c9a9dd"
format_stamp: "Formatted at 2026-08-11 06:29:27 on dist-test-slave-gkw7"
server_key: "f01588f9955967618d21a158da1042c4"
server_key_iv: "740411a4c189b1816d45a968d0737c41"
server_key_version: "encryptionkey@0"
I20260811 06:29:27.575371 31476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/minicluster-data/master-0/wal/instance:
uuid: "66e064d761374e509bbcfe0045c9a9dd"
format_stamp: "Formatted at 2026-08-11 06:29:27 on dist-test-slave-gkw7"
server_key: "f01588f9955967618d21a158da1042c4"
server_key_iv: "740411a4c189b1816d45a968d0737c41"
server_key_version: "encryptionkey@0"
I20260811 06:29:27.580873 31476 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 06:29:27.584537 31495 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:29:27.585955 31476 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:29:27.586186 31476 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/minicluster-data/master-0/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/minicluster-data/master-0/wal
uuid: "66e064d761374e509bbcfe0045c9a9dd"
format_stamp: "Formatted at 2026-08-11 06:29:27 on dist-test-slave-gkw7"
server_key: "f01588f9955967618d21a158da1042c4"
server_key_iv: "740411a4c189b1816d45a968d0737c41"
server_key_version: "encryptionkey@0"
I20260811 06:29:27.586433 31476 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:29:27.601871 31476 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:29:27.606221 31476 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:29:27.606519 31476 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:29:27.630357 31476 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.254:43627
I20260811 06:29:27.630391 31599 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.254:43627 every 8 connection(s)
I20260811 06:29:27.631832 31476 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
I20260811 06:29:27.633721 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 31476
I20260811 06:29:27.633963 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/minicluster-data/master-0/wal/instance
I20260811 06:29:27.634604 29631 external_mini_cluster.cc:1428] Setting key da3fa2d3bf734d4ba70b8b72f03a68ee
I20260811 06:29:27.636415 31600 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:29:27.650727 31600 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66e064d761374e509bbcfe0045c9a9dd: Bootstrap starting.
Aug 11 06:29:27 dist-test-slave-gkw7 krb5kdc[31458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429767, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:29:27.654837 31600 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 66e064d761374e509bbcfe0045c9a9dd: Neither blocks nor log segments found. Creating new log.
I20260811 06:29:27.656293 31600 log.cc:824] T 00000000000000000000000000000000 P 66e064d761374e509bbcfe0045c9a9dd: Log is configured to *not* fsync() on all Append() calls
I20260811 06:29:27.659405 31600 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66e064d761374e509bbcfe0045c9a9dd: No bootstrap required, opened a new log
I20260811 06:29:27.661298 31605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:27.637037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56740 (local address 127.28.239.254:43627)
0811 06:29:27.637970 (+ 933us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:27.637989 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:27.638046 (+ 57us) server_negotiation.cc:407] Connection header received
0811 06:29:27.639973 (+ 1927us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:27.640001 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:27.640555 (+ 554us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:27.641041 (+ 486us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:27.642536 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:27.644683 (+ 2147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:27.646040 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:27.647133 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:27.647321 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:27.652344 (+ 5023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:29:27.652357 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:29:27.652380 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:29:27.652440 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:29:27.656659 (+ 4219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:27.657471 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:27.657479 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:27.657487 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:27.657754 (+ 267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:27.658241 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:27.658246 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:27.658248 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:27.658846 (+ 598us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:29:27.659161 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:27.659712 (+ 551us) server_negotiation.cc:299] Negotiation successful
0811 06:29:27.660178 (+ 466us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":664,"thread_start_us":434,"threads_started":1}
I20260811 06:29:27.669296 31600 raft_consensus.cc:348] T 00000000000000000000000000000000 P 66e064d761374e509bbcfe0045c9a9dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66e064d761374e509bbcfe0045c9a9dd" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 43627 } }
I20260811 06:29:27.669941 31600 raft_consensus.cc:374] T 00000000000000000000000000000000 P 66e064d761374e509bbcfe0045c9a9dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:29:27.670112 31600 raft_consensus.cc:729] T 00000000000000000000000000000000 P 66e064d761374e509bbcfe0045c9a9dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66e064d761374e509bbcfe0045c9a9dd, State: Initialized, Role: FOLLOWER
I20260811 06:29:27.670863 31600 consensus_queue.cc:260] T 00000000000000000000000000000000 P 66e064d761374e509bbcfe0045c9a9dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66e064d761374e509bbcfe0045c9a9dd" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 43627 } }
I20260811 06:29:27.671167 31600 raft_consensus.cc:388] T 00000000000000000000000000000000 P 66e064d761374e509bbcfe0045c9a9dd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:29:27.671331 31600 raft_consensus.cc:482] T 00000000000000000000000000000000 P 66e064d761374e509bbcfe0045c9a9dd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:29:27.671561 31600 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 66e064d761374e509bbcfe0045c9a9dd [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:29:27.674281 31600 raft_consensus.cc:504] T 00000000000000000000000000000000 P 66e064d761374e509bbcfe0045c9a9dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66e064d761374e509bbcfe0045c9a9dd" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 43627 } }
I20260811 06:29:27.674911 31600 leader_election.cc:302] T 00000000000000000000000000000000 P 66e064d761374e509bbcfe0045c9a9dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 66e064d761374e509bbcfe0045c9a9dd; no voters:
I20260811 06:29:27.675776 31600 leader_election.cc:288] T 00000000000000000000000000000000 P 66e064d761374e509bbcfe0045c9a9dd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:29:27.675933 31607 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 66e064d761374e509bbcfe0045c9a9dd [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:29:27.676815 31607 raft_consensus.cc:686] T 00000000000000000000000000000000 P 66e064d761374e509bbcfe0045c9a9dd [term 1 LEADER]: Becoming Leader. State: Replica: 66e064d761374e509bbcfe0045c9a9dd, State: Running, Role: LEADER
I20260811 06:29:27.677433 31607 consensus_queue.cc:237] T 00000000000000000000000000000000 P 66e064d761374e509bbcfe0045c9a9dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66e064d761374e509bbcfe0045c9a9dd" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 43627 } }
I20260811 06:29:27.678499 31600 sys_catalog.cc:564] T 00000000000000000000000000000000 P 66e064d761374e509bbcfe0045c9a9dd [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:29:27.680708 31609 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66e064d761374e509bbcfe0045c9a9dd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 66e064d761374e509bbcfe0045c9a9dd. Latest consensus state: current_term: 1 leader_uuid: "66e064d761374e509bbcfe0045c9a9dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66e064d761374e509bbcfe0045c9a9dd" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 43627 } } }
I20260811 06:29:27.681244 31609 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66e064d761374e509bbcfe0045c9a9dd [sys.catalog]: This master's current role is: LEADER
I20260811 06:29:27.684387 31622 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:29:27.684214 31608 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66e064d761374e509bbcfe0045c9a9dd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "66e064d761374e509bbcfe0045c9a9dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66e064d761374e509bbcfe0045c9a9dd" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 43627 } } }
I20260811 06:29:27.684973 31608 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66e064d761374e509bbcfe0045c9a9dd [sys.catalog]: This master's current role is: LEADER
I20260811 06:29:27.691313 31622 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:29:27.702188 31622 catalog_manager.cc:1347] Generated new cluster ID: ceb63945429a4ed3b00ead5cc7c8adbf
I20260811 06:29:27.702329 31622 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:29:27.732378 31622 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:29:27.733924 31622 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:29:27.742573 31622 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 66e064d761374e509bbcfe0045c9a9dd: Generated new TSK 0
I20260811 06:29:27.743732 31622 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.239.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.239.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.239.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.193.keytab.
Entry for principal kudu/127.28.239.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.193.keytab.
I20260811 06:29:27.818321 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.193:0
--local_ip_for_outbound_sockets=127.28.239.193
--webserver_interface=127.28.239.193
--webserver_port=0
--tserver_master_addrs=127.28.239.254:43627
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.193.keytab
--principal=kudu/127.28.239.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:40295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:29:28.163627 31637 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:29:28.163944 31637 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:29:28.164131 31637 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:29:28.171732 31637 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:29:28.171870 31637 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:29:28.172066 31637 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 06:29:28.172202 31637 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.193
I20260811 06:29:28.182824 31637 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:40295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.193.keytab
--principal=kudu/127.28.239.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.239.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.28.239.254:43627
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.193
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:29:28.184294 31637 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:29:28.185838 31637 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:29:28.200453 31645 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:29:28.200420 31648 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:29:28.200357 31644 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:29:28.203022 31637 server_base.cc:1034] running on GCE node
I20260811 06:29:28.203835 31637 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:29:28.204895 31637 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:29:28.206156 31637 hybrid_clock.cc:648] HybridClock initialized: now 1786429768206071 us; error 65 us; skew 500 ppm
I20260811 06:29:28.206496 31637 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:29:28 dist-test-slave-gkw7 krb5kdc[31458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429768, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:29:28.215178 31637 init.cc:377] Logged in from keytab as kudu/127.28.239.193@KRBTEST.COM (short username kudu)
I20260811 06:29:28.217054 31637 webserver.cc:460] Webserver started at http://127.28.239.193:34763/ using document root <none> and password file <none>
I20260811 06:29:28.217983 31637 fs_manager.cc:359] Metadata directory not provided
I20260811 06:29:28.218132 31637 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:29:28.218487 31637 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:29:28.221658 31637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/minicluster-data/ts-0/data/instance:
uuid: "24f99d81ad37473e9da0352cfc3101b0"
format_stamp: "Formatted at 2026-08-11 06:29:28 on dist-test-slave-gkw7"
server_key: "ef1c1330260d5ec75ee7391bec383f02"
server_key_iv: "adfab919767d27f2881e375be42ca9fe"
server_key_version: "encryptionkey@0"
I20260811 06:29:28.222534 31637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/minicluster-data/ts-0/wal/instance:
uuid: "24f99d81ad37473e9da0352cfc3101b0"
format_stamp: "Formatted at 2026-08-11 06:29:28 on dist-test-slave-gkw7"
server_key: "ef1c1330260d5ec75ee7391bec383f02"
server_key_iv: "adfab919767d27f2881e375be42ca9fe"
server_key_version: "encryptionkey@0"
I20260811 06:29:28.228037 31637 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 06:29:28.231801 31662 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:29:28.233223 31637 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 06:29:28.233419 31637 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/minicluster-data/ts-0/wal
uuid: "24f99d81ad37473e9da0352cfc3101b0"
format_stamp: "Formatted at 2026-08-11 06:29:28 on dist-test-slave-gkw7"
server_key: "ef1c1330260d5ec75ee7391bec383f02"
server_key_iv: "adfab919767d27f2881e375be42ca9fe"
server_key_version: "encryptionkey@0"
I20260811 06:29:28.233685 31637 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:29:28.253201 31637 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:29:28.257807 31637 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:29:28.258105 31637 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:29:28.261186 31637 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:29:28.261288 31637 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:29:28.261432 31637 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:29:28.261523 31637 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 06:29:28 dist-test-slave-gkw7 krb5kdc[31458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429768, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.193@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:29:28.287443 31678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:28.261848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.193:44283 (local address 127.28.239.254:43627)
0811 06:29:28.262549 (+ 701us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:28.262554 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:28.263638 (+ 1084us) server_negotiation.cc:407] Connection header received
0811 06:29:28.265815 (+ 2177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:28.265822 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:28.265918 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:28.266056 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:28.269323 (+ 3267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:28.270627 (+ 1304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:28.273338 (+ 2711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:28.274116 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:28.274298 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:28.279591 (+ 5293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:29:28.279598 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:29:28.279605 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:29:28.279682 (+ 77us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:29:28.283104 (+ 3422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:28.284412 (+ 1308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:28.284419 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:28.284425 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:28.284593 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:28.285302 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:28.285309 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:28.285313 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:28.285742 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:29:28.285887 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:28.286813 (+ 926us) server_negotiation.cc:299] Negotiation successful
0811 06:29:28.287043 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":587,"thread_start_us":490,"threads_started":1}
I20260811 06:29:28.288582 31679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:28.262576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:43627 (local address 127.28.239.193:44283)
0811 06:29:28.263464 (+ 888us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:28.263502 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 06:29:28.265453 (+ 1951us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:29:28.266585 (+ 1132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:29:28.266605 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:29:28.267669 (+ 1064us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:29:28.269025 (+ 1356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:28.269056 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:28.270863 (+ 1807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:28.270871 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:28.273080 (+ 2209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:28.273093 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:28.274357 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:28.274362 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:28.274734 (+ 372us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:28.275630 (+ 896us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:29:28.275657 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:29:28.279306 (+ 3649us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:29:28.283330 (+ 4024us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:29:28.283346 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:29:28.283376 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:29:28.284229 (+ 853us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:29:28.284730 (+ 501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:29:28.284735 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:29:28.284739 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:29:28.285181 (+ 442us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:29:28.286004 (+ 823us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:29:28.286017 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:29:28.286425 (+ 408us) client_negotiation.cc:769] Sending connection context
0811 06:29:28.286717 (+ 292us) client_negotiation.cc:240] Negotiation successful
0811 06:29:28.287300 (+ 583us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":640,"thread_start_us":417,"threads_started":1}
I20260811 06:29:28.309713 31790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:28.295744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:43627 (local address 127.28.239.193:60403)
0811 06:29:28.296426 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:28.296442 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:29:28.296555 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:29:28.296951 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:29:28.296955 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:29:28.297439 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:29:28.297862 (+ 423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:28.297873 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:28.299091 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:28.299095 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:28.300114 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:28.300125 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:28.300903 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:28.300909 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:28.301093 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:28.301791 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:29:28.301813 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:29:28.303136 (+ 1323us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:29:28.306408 (+ 3272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:29:28.306417 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:29:28.306421 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:29:28.307104 (+ 683us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:29:28.307499 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:29:28.307504 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:29:28.307508 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:29:28.307716 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:29:28.308521 (+ 805us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:29:28.308528 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:29:28.308742 (+ 214us) client_negotiation.cc:769] Sending connection context
0811 06:29:28.308977 (+ 235us) client_negotiation.cc:240] Negotiation successful
0811 06:29:28.309298 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":528,"thread_start_us":367,"threads_started":1}
I20260811 06:29:28.310010 31678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:28.295627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.193:60403 (local address 127.28.239.254:43627)
0811 06:29:28.295861 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:28.295869 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:28.296529 (+ 660us) server_negotiation.cc:407] Connection header received
0811 06:29:28.296669 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:28.296677 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:28.296795 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:28.296936 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:28.298016 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:28.298937 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:28.300254 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:28.300711 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:28.300854 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:28.303382 (+ 2528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:29:28.303388 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:29:28.303393 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:29:28.303430 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:29:28.306264 (+ 2834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:28.307236 (+ 972us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:28.307244 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:28.307249 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:28.307379 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:28.307882 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:28.307888 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:28.307892 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:28.308235 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:29:28.308394 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:28.309126 (+ 732us) server_negotiation.cc:299] Negotiation successful
0811 06:29:28.309468 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 06:29:28.319881 31637 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.193:41407
I20260811 06:29:28.319909 31863 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.193:41407 every 8 connection(s)
I20260811 06:29:28.321185 31637 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
I20260811 06:29:28.323547 31864 heartbeater.cc:347] Connected to a master server at 127.28.239.254:43627
I20260811 06:29:28.323812 31864 heartbeater.cc:464] Registering TS with master...
I20260811 06:29:28.324487 31864 heartbeater.cc:511] Master 127.28.239.254:43627 requested a full tablet report, sending...
I20260811 06:29:28.326555 31518 ts_manager.cc:194] Registered new tserver with Master: 24f99d81ad37473e9da0352cfc3101b0 (127.28.239.193:41407)
I20260811 06:29:28.328073 31518 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.239.193@KRBTEST.COM'} at 127.28.239.193:44283
I20260811 06:29:28.331264 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 31637
I20260811 06:29:28.331454 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/minicluster-data/ts-0/wal/instance
I20260811 06:29:28.332018 29631 external_mini_cluster.cc:1428] Setting key c536391a0c2774ed74cd1331c6121528
WARNING: no policy specified for kudu/127.28.239.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.239.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.239.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.194.keytab.
Entry for principal kudu/127.28.239.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.194.keytab.
I20260811 06:29:28.374945 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.194:0
--local_ip_for_outbound_sockets=127.28.239.194
--webserver_interface=127.28.239.194
--webserver_port=0
--tserver_master_addrs=127.28.239.254:43627
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.194.keytab
--principal=kudu/127.28.239.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:40295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:29:28.713927 31869 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:29:28.714273 31869 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:29:28.714445 31869 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:29:28.721861 31869 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:29:28.722030 31869 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:29:28.722189 31869 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 06:29:28.722330 31869 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.194
I20260811 06:29:28.732569 31869 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:40295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.194.keytab
--principal=kudu/127.28.239.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.239.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.28.239.254:43627
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.194
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:29:28.734014 31869 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:29:28.735446 31869 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:29:28.749177 31881 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:29:28.749348 31878 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:29:28.749272 31879 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:29:28.750486 31869 server_base.cc:1034] running on GCE node
I20260811 06:29:28.751205 31869 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:29:28.752179 31869 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:29:28.753453 31869 hybrid_clock.cc:648] HybridClock initialized: now 1786429768753385 us; error 36 us; skew 500 ppm
I20260811 06:29:28.753834 31869 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:29:28 dist-test-slave-gkw7 krb5kdc[31458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429768, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:29:28.757141 31869 init.cc:377] Logged in from keytab as kudu/127.28.239.194@KRBTEST.COM (short username kudu)
I20260811 06:29:28.758682 31869 webserver.cc:460] Webserver started at http://127.28.239.194:46603/ using document root <none> and password file <none>
I20260811 06:29:28.759377 31869 fs_manager.cc:359] Metadata directory not provided
I20260811 06:29:28.759467 31869 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:29:28.759743 31869 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:29:28.762737 31869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/minicluster-data/ts-1/data/instance:
uuid: "6faa0ae89fb54a1eafe2f27278e53c89"
format_stamp: "Formatted at 2026-08-11 06:29:28 on dist-test-slave-gkw7"
server_key: "9f30a62970dcbfcb47e4dd482e7de7d9"
server_key_iv: "f9848415fc3c38ebbac421b50e447d22"
server_key_version: "encryptionkey@0"
I20260811 06:29:28.763515 31869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/minicluster-data/ts-1/wal/instance:
uuid: "6faa0ae89fb54a1eafe2f27278e53c89"
format_stamp: "Formatted at 2026-08-11 06:29:28 on dist-test-slave-gkw7"
server_key: "9f30a62970dcbfcb47e4dd482e7de7d9"
server_key_iv: "f9848415fc3c38ebbac421b50e447d22"
server_key_version: "encryptionkey@0"
I20260811 06:29:28.768838 31869 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 06:29:28.772773 31889 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:29:28.774325 31869 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 06:29:28.774456 31869 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/minicluster-data/ts-1/wal
uuid: "6faa0ae89fb54a1eafe2f27278e53c89"
format_stamp: "Formatted at 2026-08-11 06:29:28 on dist-test-slave-gkw7"
server_key: "9f30a62970dcbfcb47e4dd482e7de7d9"
server_key_iv: "f9848415fc3c38ebbac421b50e447d22"
server_key_version: "encryptionkey@0"
I20260811 06:29:28.774632 31869 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:29:28.799026 31869 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:29:28.803287 31869 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:29:28.803534 31869 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:29:28.806468 31869 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:29:28.806576 31869 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:29:28.806705 31869 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:29:28.806785 31869 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 06:29:28 dist-test-slave-gkw7 krb5kdc[31458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429768, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.194@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:29:28.836858 31908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:28.807926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:43627 (local address 127.28.239.194:54711)
0811 06:29:28.808863 (+ 937us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:28.808919 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 06:29:28.811630 (+ 2711us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:29:28.812746 (+ 1116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:29:28.812761 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:29:28.813731 (+ 970us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:29:28.814856 (+ 1125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:28.814912 (+ 56us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:28.816532 (+ 1620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:28.816536 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:28.818941 (+ 2405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:28.818986 (+ 45us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:28.820382 (+ 1396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:28.820394 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:28.820768 (+ 374us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:28.821826 (+ 1058us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:29:28.821876 (+ 50us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:29:28.825497 (+ 3621us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:29:28.830206 (+ 4709us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:29:28.830217 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:29:28.830248 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:29:28.831017 (+ 769us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:29:28.831630 (+ 613us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:29:28.831639 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:29:28.831643 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:29:28.832146 (+ 503us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:29:28.832978 (+ 832us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:29:28.832988 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:29:28.833712 (+ 724us) client_negotiation.cc:769] Sending connection context
0811 06:29:28.834130 (+ 418us) client_negotiation.cc:240] Negotiation successful
0811 06:29:28.834828 (+ 698us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":667,"thread_start_us":405,"threads_started":1}
I20260811 06:29:28.838186 31678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:28.807185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.194:54711 (local address 127.28.239.254:43627)
0811 06:29:28.807412 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:28.807419 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:28.809091 (+ 1672us) server_negotiation.cc:407] Connection header received
0811 06:29:28.812106 (+ 3015us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:28.812116 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:28.812231 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:28.812375 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:28.815139 (+ 2764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:28.816285 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:28.819176 (+ 2891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:28.820036 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:28.820199 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:28.825833 (+ 5634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:29:28.825838 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:29:28.825842 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:29:28.825882 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:29:28.830049 (+ 4167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:28.831274 (+ 1225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:28.831284 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:28.831288 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:28.831471 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:28.832335 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:28.832339 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:28.832342 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:28.832820 (+ 478us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:29:28.833018 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:28.837606 (+ 4588us) server_negotiation.cc:299] Negotiation successful
0811 06:29:28.837837 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 06:29:28.855489 31869 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.194:37745
I20260811 06:29:28.855508 32086 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.194:37745 every 8 connection(s)
I20260811 06:29:28.856819 31869 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
I20260811 06:29:28.859236 32087 heartbeater.cc:347] Connected to a master server at 127.28.239.254:43627
I20260811 06:29:28.859508 32087 heartbeater.cc:464] Registering TS with master...
I20260811 06:29:28.860384 32087 heartbeater.cc:511] Master 127.28.239.254:43627 requested a full tablet report, sending...
I20260811 06:29:28.862028 31518 ts_manager.cc:194] Registered new tserver with Master: 6faa0ae89fb54a1eafe2f27278e53c89 (127.28.239.194:37745)
I20260811 06:29:28.862975 31518 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.239.194@KRBTEST.COM'} at 127.28.239.194:54711
I20260811 06:29:28.864212 31678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:28.844279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.194:56785 (local address 127.28.239.254:43627)
0811 06:29:28.844546 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:28.844555 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:28.845068 (+ 513us) server_negotiation.cc:407] Connection header received
0811 06:29:28.845310 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:28.845315 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:28.845406 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:28.845518 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:28.846898 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:28.847697 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:28.850511 (+ 2814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:28.851032 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:28.851202 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:28.853912 (+ 2710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:29:28.853918 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:29:28.853921 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:29:28.853961 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:29:28.861223 (+ 7262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:28.862001 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:28.862006 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:28.862010 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:28.862224 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:28.862633 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:28.862639 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:28.862642 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:28.863037 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:29:28.863197 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:28.863615 (+ 418us) server_negotiation.cc:299] Negotiation successful
0811 06:29:28.863840 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":16,"server-negotiator.queue_time_us":79}
I20260811 06:29:28.864341 32045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:28.844195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:43627 (local address 127.28.239.194:56785)
0811 06:29:28.844916 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:28.844939 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 06:29:28.845135 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:29:28.845629 (+ 494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:29:28.845637 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:29:28.846409 (+ 772us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:29:28.846734 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:28.846745 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:28.847864 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:28.847873 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:28.850251 (+ 2378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:28.850281 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:28.851472 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:28.851477 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:28.851602 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:28.852512 (+ 910us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:29:28.852548 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:29:28.853724 (+ 1176us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:29:28.861377 (+ 7653us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:29:28.861383 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:29:28.861387 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:29:28.861869 (+ 482us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:29:28.862365 (+ 496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:29:28.862371 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:29:28.862374 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:29:28.862511 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:29:28.863162 (+ 651us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:29:28.863167 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:29:28.863355 (+ 188us) client_negotiation.cc:769] Sending connection context
0811 06:29:28.863495 (+ 140us) client_negotiation.cc:240] Negotiation successful
0811 06:29:28.863857 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":353,"threads_started":1}
I20260811 06:29:28.865854 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 31869
I20260811 06:29:28.866060 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/minicluster-data/ts-1/wal/instance
I20260811 06:29:28.866472 29631 external_mini_cluster.cc:1428] Setting key b51a8c035af695e16dcef7620457cdf3
WARNING: no policy specified for kudu/127.28.239.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.239.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.239.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.195.keytab.
Entry for principal kudu/127.28.239.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.195.keytab.
I20260811 06:29:28.916849 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.195:0
--local_ip_for_outbound_sockets=127.28.239.195
--webserver_interface=127.28.239.195
--webserver_port=0
--tserver_master_addrs=127.28.239.254:43627
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.195.keytab
--principal=kudu/127.28.239.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:40295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:29:29.283028 32093 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:29:29.283330 32093 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:29:29.283495 32093 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:29:29.291110 32093 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:29:29.291296 32093 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:29:29.291445 32093 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 06:29:29.291569 32093 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.195
I20260811 06:29:29.301807 32093 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:40295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.195.keytab
--principal=kudu/127.28.239.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.239.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.28.239.254:43627
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.195
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:29:29.303213 32093 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:29:29.304625 32093 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:29:29.319212 32099 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:29:29.329818 32100 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:29:29.329866 32103 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:29:29.329988 32093 server_base.cc:1034] running on GCE node
I20260811 06:29:29.330790 32093 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:29:29.331192 31864 heartbeater.cc:503] Master 127.28.239.254:43627 was elected leader, sending a full tablet report...
I20260811 06:29:29.332142 32093 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:29:29.333398 32093 hybrid_clock.cc:648] HybridClock initialized: now 1786429769333340 us; error 36 us; skew 500 ppm
I20260811 06:29:29.333781 32093 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:29:29 dist-test-slave-gkw7 krb5kdc[31458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429769, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:29:29.337100 32093 init.cc:377] Logged in from keytab as kudu/127.28.239.195@KRBTEST.COM (short username kudu)
I20260811 06:29:29.339164 32093 webserver.cc:460] Webserver started at http://127.28.239.195:45523/ using document root <none> and password file <none>
I20260811 06:29:29.340003 32093 fs_manager.cc:359] Metadata directory not provided
I20260811 06:29:29.340142 32093 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:29:29.340514 32093 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:29:29.343799 32093 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/minicluster-data/ts-2/data/instance:
uuid: "a3ef849ee1e54c1b9e0fc8ba8a8f267c"
format_stamp: "Formatted at 2026-08-11 06:29:29 on dist-test-slave-gkw7"
server_key: "f8c2888ac936a467b1d150a706709127"
server_key_iv: "22ce5b7317a1972ec4b131e8f148fce2"
server_key_version: "encryptionkey@0"
I20260811 06:29:29.344636 32093 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/minicluster-data/ts-2/wal/instance:
uuid: "a3ef849ee1e54c1b9e0fc8ba8a8f267c"
format_stamp: "Formatted at 2026-08-11 06:29:29 on dist-test-slave-gkw7"
server_key: "f8c2888ac936a467b1d150a706709127"
server_key_iv: "22ce5b7317a1972ec4b131e8f148fce2"
server_key_version: "encryptionkey@0"
I20260811 06:29:29.350132 32093 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 06:29:29.353979 32112 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:29:29.355404 32093 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 06:29:29.355588 32093 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/minicluster-data/ts-2/wal
uuid: "a3ef849ee1e54c1b9e0fc8ba8a8f267c"
format_stamp: "Formatted at 2026-08-11 06:29:29 on dist-test-slave-gkw7"
server_key: "f8c2888ac936a467b1d150a706709127"
server_key_iv: "22ce5b7317a1972ec4b131e8f148fce2"
server_key_version: "encryptionkey@0"
I20260811 06:29:29.355810 32093 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:29:29.380090 32093 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:29:29.385012 32093 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:29:29.385331 32093 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:29:29.388355 32093 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:29:29.388478 32093 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:29:29.388640 32093 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:29:29.388722 32093 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 06:29:29 dist-test-slave-gkw7 krb5kdc[31458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429769, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.195@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:29:29.416076 32129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:29.389359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.195:56129 (local address 127.28.239.254:43627)
0811 06:29:29.389921 (+ 562us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:29.389928 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:29.391752 (+ 1824us) server_negotiation.cc:407] Connection header received
0811 06:29:29.395408 (+ 3656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:29.395446 (+ 38us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:29.395592 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:29.395740 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:29.397770 (+ 2030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:29.398967 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:29.400890 (+ 1923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:29.401372 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:29.401563 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:29.406490 (+ 4927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:29:29.406496 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:29:29.406499 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:29:29.406538 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:29:29.411130 (+ 4592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:29.412654 (+ 1524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:29.412662 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:29.412668 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:29.412882 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:29.413714 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:29.413727 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:29.413730 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:29.414200 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:29:29.414385 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:29.415344 (+ 959us) server_negotiation.cc:299] Negotiation successful
0811 06:29:29.415631 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":422,"thread_start_us":295,"threads_started":1}
I20260811 06:29:29.417299 32130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:29.390313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:43627 (local address 127.28.239.195:56129)
0811 06:29:29.391496 (+ 1183us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:29.391582 (+ 86us) client_negotiation.cc:174] Beginning negotiation
0811 06:29:29.394927 (+ 3345us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:29:29.396074 (+ 1147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:29:29.396088 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:29:29.396859 (+ 771us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:29:29.397498 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:29.397516 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:29.399148 (+ 1632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:29.399155 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:29.400714 (+ 1559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:29.400729 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:29.401689 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:29.401698 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:29.401960 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:29.402820 (+ 860us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:29:29.402865 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:29:29.406196 (+ 3331us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:29:29.411419 (+ 5223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:29:29.411439 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:29:29.411468 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:29:29.412363 (+ 895us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:29:29.413028 (+ 665us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:29:29.413041 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:29:29.413045 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:29:29.413460 (+ 415us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:29:29.414358 (+ 898us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:29:29.414368 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:29:29.414878 (+ 510us) client_negotiation.cc:769] Sending connection context
0811 06:29:29.415215 (+ 337us) client_negotiation.cc:240] Negotiation successful
0811 06:29:29.415859 (+ 644us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":843,"thread_start_us":545,"threads_started":1}
I20260811 06:29:29.450575 32129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:29.429466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.195:43179 (local address 127.28.239.254:43627)
0811 06:29:29.429845 (+ 379us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:29.429853 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:29.430916 (+ 1063us) server_negotiation.cc:407] Connection header received
0811 06:29:29.431278 (+ 362us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:29.431284 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:29.431394 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:29.431481 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:29.433469 (+ 1988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:29.434471 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:29.437680 (+ 3209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:29.438448 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:29.438637 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:29.442137 (+ 3500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:29:29.442144 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:29:29.442147 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:29:29.442187 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:29:29.446725 (+ 4538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:29.447771 (+ 1046us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:29.447778 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:29.447783 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:29.447992 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:29.448606 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:29.448614 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:29.448620 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:29.448995 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:29:29.449115 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:29.449850 (+ 735us) server_negotiation.cc:299] Negotiation successful
0811 06:29:29.450131 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138}
I20260811 06:29:29.451749 32229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:29.429478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:43627 (local address 127.28.239.195:43179)
0811 06:29:29.430776 (+ 1298us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:29.430804 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0811 06:29:29.431021 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:29:29.431719 (+ 698us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:29:29.431723 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:29:29.432688 (+ 965us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:29:29.433273 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:29.433292 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:29.434693 (+ 1401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:29.434704 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:29.437356 (+ 2652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:29.437385 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:29.438731 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:29.438743 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:29.439032 (+ 289us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:29.440150 (+ 1118us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:29:29.440184 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:29:29.441891 (+ 1707us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:29:29.446934 (+ 5043us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:29:29.446944 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:29:29.446950 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:29:29.447573 (+ 623us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:29:29.448204 (+ 631us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:29:29.448212 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:29:29.448220 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:29:29.448444 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:29:29.449154 (+ 710us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:29:29.449162 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:29:29.449413 (+ 251us) client_negotiation.cc:769] Sending connection context
0811 06:29:29.449685 (+ 272us) client_negotiation.cc:240] Negotiation successful
0811 06:29:29.450496 (+ 811us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1107,"thread_start_us":885,"threads_started":1}
I20260811 06:29:29.454864 32093 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.195:34897
I20260811 06:29:29.454888 32292 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.195:34897 every 8 connection(s)
I20260811 06:29:29.456357 32093 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
I20260811 06:29:29.458977 32295 heartbeater.cc:347] Connected to a master server at 127.28.239.254:43627
I20260811 06:29:29.459326 32295 heartbeater.cc:464] Registering TS with master...
I20260811 06:29:29.460274 32295 heartbeater.cc:511] Master 127.28.239.254:43627 requested a full tablet report, sending...
I20260811 06:29:29.462080 31518 ts_manager.cc:194] Registered new tserver with Master: a3ef849ee1e54c1b9e0fc8ba8a8f267c (127.28.239.195:34897)
I20260811 06:29:29.463294 31518 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.239.195@KRBTEST.COM'} at 127.28.239.195:56129
I20260811 06:29:29.466033 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 32093
I20260811 06:29:29.466253 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/minicluster-data/ts-2/wal/instance
I20260811 06:29:29.466817 29631 external_mini_cluster.cc:1428] Setting key d2e8a2a0e31c8e4d9bfb7a8d2c5abb0d
I20260811 06:29:29.470650 29631 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 06:29:29 dist-test-slave-gkw7 krb5kdc[31458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429769, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 06:29:29 dist-test-slave-gkw7 krb5kdc[31458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429769, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:29:29.526185 32129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:29.511658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56750 (local address 127.28.239.254:43627)
0811 06:29:29.511841 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:29.511846 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:29.512163 (+ 317us) server_negotiation.cc:407] Connection header received
0811 06:29:29.512378 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:29.512386 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:29.512483 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:29.512619 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:29.514082 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:29.514840 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:29.516138 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:29.516540 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:29.516639 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:29.520804 (+ 4165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:29:29.520809 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:29:29.520812 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:29:29.520850 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:29:29.523434 (+ 2584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:29.524113 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:29.524119 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:29.524122 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:29.524279 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:29.524668 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:29.524673 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:29.524678 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:29.524964 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:29:29.525251 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:29.525587 (+ 336us) server_negotiation.cc:299] Negotiation successful
0811 06:29:29.525814 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 06:29:29.530848 31518 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56750:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:29:29.538888 31518 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:29:29.575829 32314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:29.552219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.193:41407 (local address 127.0.0.1:59594)
0811 06:29:29.566321 (+ 14102us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:29.566345 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 06:29:29.566475 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:29:29.567140 (+ 665us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:29:29.567149 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:29:29.567178 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:29:29.567636 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:29.567650 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:29.569664 (+ 2014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:29.569670 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:29.572151 (+ 2481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:29.572165 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:29.574415 (+ 2250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:29.574422 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:29.574656 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:29.574734 (+ 78us) client_negotiation.cc:769] Sending connection context
0811 06:29:29.574849 (+ 115us) client_negotiation.cc:240] Negotiation successful
0811 06:29:29.575295 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13951,"mutex_wait_us":396,"thread_start_us":12074,"threads_started":1}
I20260811 06:29:29.575946 32319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:29.552514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59594 (local address 127.28.239.193:41407)
0811 06:29:29.565985 (+ 13471us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:29.565998 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:29.566612 (+ 614us) server_negotiation.cc:407] Connection header received
0811 06:29:29.566719 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:29.566739 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:29.566918 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:29.567132 (+ 214us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:29:29.567844 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:29.569512 (+ 1668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:29.572331 (+ 2819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:29.574173 (+ 1842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:29.575153 (+ 980us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:29.575337 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:29.575412 (+ 75us) server_negotiation.cc:299] Negotiation successful
0811 06:29:29.575594 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13286,"thread_start_us":13140,"threads_started":1}
I20260811 06:29:29.576567 32316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:29.552142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58532 (local address 127.28.239.194:37745)
0811 06:29:29.566061 (+ 13919us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:29.566075 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:29.566115 (+ 40us) server_negotiation.cc:407] Connection header received
0811 06:29:29.566257 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:29.566279 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:29.566605 (+ 326us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:29.567374 (+ 769us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:29:29.568588 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:29.570324 (+ 1736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:29.573107 (+ 2783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:29.574415 (+ 1308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:29.574586 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:29.574780 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:29.576097 (+ 1317us) server_negotiation.cc:299] Negotiation successful
0811 06:29:29.576274 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13685,"thread_start_us":13869,"threads_started":1}
I20260811 06:29:29.577555 32318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:29.551967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.194:37745 (local address 127.0.0.1:58532)
0811 06:29:29.564891 (+ 12924us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:29.564969 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0811 06:29:29.565274 (+ 305us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:29:29.567923 (+ 2649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:29:29.567930 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:29:29.567958 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:29:29.568369 (+ 411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:29.568382 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:29.570525 (+ 2143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:29.570532 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:29.572945 (+ 2413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:29.572958 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:29.574945 (+ 1987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:29.574954 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:29.575412 (+ 458us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:29.575454 (+ 42us) client_negotiation.cc:769] Sending connection context
0811 06:29:29.575547 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 06:29:29.575683 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12514,"mutex_wait_us":249,"thread_start_us":14267,"threads_started":1}
I20260811 06:29:29.578451 32321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:29.555628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50570 (local address 127.28.239.195:34897)
0811 06:29:29.568310 (+ 12682us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:29.568323 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:29.568943 (+ 620us) server_negotiation.cc:407] Connection header received
0811 06:29:29.569147 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:29.569156 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:29.569449 (+ 293us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:29.569674 (+ 225us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:29:29.570611 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:29.572356 (+ 1745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:29.575292 (+ 2936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:29.577031 (+ 1739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:29.577260 (+ 229us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:29.577419 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:29.577925 (+ 506us) server_negotiation.cc:299] Negotiation successful
0811 06:29:29.578101 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12461,"thread_start_us":12270,"threads_started":1}
I20260811 06:29:29.578886 32320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:29.554741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.195:34897 (local address 127.0.0.1:50570)
0811 06:29:29.568808 (+ 14067us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:29.568823 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 06:29:29.568972 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:29:29.569759 (+ 787us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:29:29.569764 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:29:29.569792 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:29:29.570268 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:29.570282 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:29.572499 (+ 2217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:29.572504 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:29.575107 (+ 2603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:29.575124 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:29.577249 (+ 2125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:29.577256 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:29.577452 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:29.577484 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 06:29:29.577642 (+ 158us) client_negotiation.cc:240] Negotiation successful
0811 06:29:29.577798 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13936,"mutex_wait_us":1,"thread_start_us":13778,"threads_started":1}
I20260811 06:29:29.580495 31752 tablet_service.cc:1467] Processing CreateTablet for tablet 2e82aa7807dc4aeda8bdbebd93818db1 (DEFAULT_TABLE table=test-table [id=c329d285d6784d55b2fdcc9323e0065a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:29:29.581349 31977 tablet_service.cc:1467] Processing CreateTablet for tablet 2e82aa7807dc4aeda8bdbebd93818db1 (DEFAULT_TABLE table=test-table [id=c329d285d6784d55b2fdcc9323e0065a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:29:29.581984 31752 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e82aa7807dc4aeda8bdbebd93818db1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:29:29.582831 31977 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e82aa7807dc4aeda8bdbebd93818db1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:29:29.582492 32193 tablet_service.cc:1467] Processing CreateTablet for tablet 2e82aa7807dc4aeda8bdbebd93818db1 (DEFAULT_TABLE table=test-table [id=c329d285d6784d55b2fdcc9323e0065a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:29:29.584285 32193 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e82aa7807dc4aeda8bdbebd93818db1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:29:29.594218 32322 tablet_bootstrap.cc:492] T 2e82aa7807dc4aeda8bdbebd93818db1 P 24f99d81ad37473e9da0352cfc3101b0: Bootstrap starting.
I20260811 06:29:29.598039 32322 tablet_bootstrap.cc:654] T 2e82aa7807dc4aeda8bdbebd93818db1 P 24f99d81ad37473e9da0352cfc3101b0: Neither blocks nor log segments found. Creating new log.
I20260811 06:29:29.598047 32327 tablet_bootstrap.cc:492] T 2e82aa7807dc4aeda8bdbebd93818db1 P 6faa0ae89fb54a1eafe2f27278e53c89: Bootstrap starting.
I20260811 06:29:29.599673 32322 log.cc:824] T 2e82aa7807dc4aeda8bdbebd93818db1 P 24f99d81ad37473e9da0352cfc3101b0: Log is configured to *not* fsync() on all Append() calls
I20260811 06:29:29.601972 32327 tablet_bootstrap.cc:654] T 2e82aa7807dc4aeda8bdbebd93818db1 P 6faa0ae89fb54a1eafe2f27278e53c89: Neither blocks nor log segments found. Creating new log.
I20260811 06:29:29.603439 32322 tablet_bootstrap.cc:492] T 2e82aa7807dc4aeda8bdbebd93818db1 P 24f99d81ad37473e9da0352cfc3101b0: No bootstrap required, opened a new log
I20260811 06:29:29.603382 32327 log.cc:824] T 2e82aa7807dc4aeda8bdbebd93818db1 P 6faa0ae89fb54a1eafe2f27278e53c89: Log is configured to *not* fsync() on all Append() calls
I20260811 06:29:29.603734 32322 ts_tablet_manager.cc:1397] T 2e82aa7807dc4aeda8bdbebd93818db1 P 24f99d81ad37473e9da0352cfc3101b0: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.003s
I20260811 06:29:29.605738 32330 tablet_bootstrap.cc:492] T 2e82aa7807dc4aeda8bdbebd93818db1 P a3ef849ee1e54c1b9e0fc8ba8a8f267c: Bootstrap starting.
I20260811 06:29:29.606541 32327 tablet_bootstrap.cc:492] T 2e82aa7807dc4aeda8bdbebd93818db1 P 6faa0ae89fb54a1eafe2f27278e53c89: No bootstrap required, opened a new log
I20260811 06:29:29.606818 32327 ts_tablet_manager.cc:1397] T 2e82aa7807dc4aeda8bdbebd93818db1 P 6faa0ae89fb54a1eafe2f27278e53c89: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260811 06:29:29.609313 32330 tablet_bootstrap.cc:654] T 2e82aa7807dc4aeda8bdbebd93818db1 P a3ef849ee1e54c1b9e0fc8ba8a8f267c: Neither blocks nor log segments found. Creating new log.
I20260811 06:29:29.610476 32330 log.cc:824] T 2e82aa7807dc4aeda8bdbebd93818db1 P a3ef849ee1e54c1b9e0fc8ba8a8f267c: Log is configured to *not* fsync() on all Append() calls
I20260811 06:29:29.613406 32330 tablet_bootstrap.cc:492] T 2e82aa7807dc4aeda8bdbebd93818db1 P a3ef849ee1e54c1b9e0fc8ba8a8f267c: No bootstrap required, opened a new log
I20260811 06:29:29.613766 32330 ts_tablet_manager.cc:1397] T 2e82aa7807dc4aeda8bdbebd93818db1 P a3ef849ee1e54c1b9e0fc8ba8a8f267c: Time spent bootstrapping tablet: real 0.013s user 0.008s sys 0.000s
I20260811 06:29:29.615053 32322 raft_consensus.cc:348] T 2e82aa7807dc4aeda8bdbebd93818db1 P 24f99d81ad37473e9da0352cfc3101b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6faa0ae89fb54a1eafe2f27278e53c89" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37745 } } peers { permanent_uuid: "24f99d81ad37473e9da0352cfc3101b0" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 41407 } } peers { permanent_uuid: "a3ef849ee1e54c1b9e0fc8ba8a8f267c" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 34897 } }
I20260811 06:29:29.615537 32322 raft_consensus.cc:374] T 2e82aa7807dc4aeda8bdbebd93818db1 P 24f99d81ad37473e9da0352cfc3101b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:29:29.615648 32322 raft_consensus.cc:729] T 2e82aa7807dc4aeda8bdbebd93818db1 P 24f99d81ad37473e9da0352cfc3101b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24f99d81ad37473e9da0352cfc3101b0, State: Initialized, Role: FOLLOWER
I20260811 06:29:29.616349 32322 consensus_queue.cc:260] T 2e82aa7807dc4aeda8bdbebd93818db1 P 24f99d81ad37473e9da0352cfc3101b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6faa0ae89fb54a1eafe2f27278e53c89" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37745 } } peers { permanent_uuid: "24f99d81ad37473e9da0352cfc3101b0" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 41407 } } peers { permanent_uuid: "a3ef849ee1e54c1b9e0fc8ba8a8f267c" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 34897 } }
I20260811 06:29:29.618366 32322 ts_tablet_manager.cc:1428] T 2e82aa7807dc4aeda8bdbebd93818db1 P 24f99d81ad37473e9da0352cfc3101b0: Time spent starting tablet: real 0.014s user 0.012s sys 0.004s
I20260811 06:29:29.619145 32327 raft_consensus.cc:348] T 2e82aa7807dc4aeda8bdbebd93818db1 P 6faa0ae89fb54a1eafe2f27278e53c89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6faa0ae89fb54a1eafe2f27278e53c89" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37745 } } peers { permanent_uuid: "24f99d81ad37473e9da0352cfc3101b0" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 41407 } } peers { permanent_uuid: "a3ef849ee1e54c1b9e0fc8ba8a8f267c" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 34897 } }
I20260811 06:29:29.619784 32327 raft_consensus.cc:374] T 2e82aa7807dc4aeda8bdbebd93818db1 P 6faa0ae89fb54a1eafe2f27278e53c89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:29:29.619951 32327 raft_consensus.cc:729] T 2e82aa7807dc4aeda8bdbebd93818db1 P 6faa0ae89fb54a1eafe2f27278e53c89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6faa0ae89fb54a1eafe2f27278e53c89, State: Initialized, Role: FOLLOWER
I20260811 06:29:29.620822 32327 consensus_queue.cc:260] T 2e82aa7807dc4aeda8bdbebd93818db1 P 6faa0ae89fb54a1eafe2f27278e53c89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6faa0ae89fb54a1eafe2f27278e53c89" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37745 } } peers { permanent_uuid: "24f99d81ad37473e9da0352cfc3101b0" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 41407 } } peers { permanent_uuid: "a3ef849ee1e54c1b9e0fc8ba8a8f267c" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 34897 } }
I20260811 06:29:29.622371 32087 heartbeater.cc:503] Master 127.28.239.254:43627 was elected leader, sending a full tablet report...
I20260811 06:29:29.622829 32327 ts_tablet_manager.cc:1428] T 2e82aa7807dc4aeda8bdbebd93818db1 P 6faa0ae89fb54a1eafe2f27278e53c89: Time spent starting tablet: real 0.016s user 0.010s sys 0.005s
I20260811 06:29:29.625128 32330 raft_consensus.cc:348] T 2e82aa7807dc4aeda8bdbebd93818db1 P a3ef849ee1e54c1b9e0fc8ba8a8f267c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6faa0ae89fb54a1eafe2f27278e53c89" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37745 } } peers { permanent_uuid: "24f99d81ad37473e9da0352cfc3101b0" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 41407 } } peers { permanent_uuid: "a3ef849ee1e54c1b9e0fc8ba8a8f267c" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 34897 } }
I20260811 06:29:29.625648 32330 raft_consensus.cc:374] T 2e82aa7807dc4aeda8bdbebd93818db1 P a3ef849ee1e54c1b9e0fc8ba8a8f267c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:29:29.625769 32330 raft_consensus.cc:729] T 2e82aa7807dc4aeda8bdbebd93818db1 P a3ef849ee1e54c1b9e0fc8ba8a8f267c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3ef849ee1e54c1b9e0fc8ba8a8f267c, State: Initialized, Role: FOLLOWER
I20260811 06:29:29.626482 32330 consensus_queue.cc:260] T 2e82aa7807dc4aeda8bdbebd93818db1 P a3ef849ee1e54c1b9e0fc8ba8a8f267c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6faa0ae89fb54a1eafe2f27278e53c89" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37745 } } peers { permanent_uuid: "24f99d81ad37473e9da0352cfc3101b0" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 41407 } } peers { permanent_uuid: "a3ef849ee1e54c1b9e0fc8ba8a8f267c" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 34897 } }
I20260811 06:29:29.627702 32295 heartbeater.cc:503] Master 127.28.239.254:43627 was elected leader, sending a full tablet report...
I20260811 06:29:29.628238 32330 ts_tablet_manager.cc:1428] T 2e82aa7807dc4aeda8bdbebd93818db1 P a3ef849ee1e54c1b9e0fc8ba8a8f267c: Time spent starting tablet: real 0.014s user 0.008s sys 0.004s
I20260811 06:29:29.682050 32338 raft_consensus.cc:482] T 2e82aa7807dc4aeda8bdbebd93818db1 P a3ef849ee1e54c1b9e0fc8ba8a8f267c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:29:29.682438 32338 raft_consensus.cc:504] T 2e82aa7807dc4aeda8bdbebd93818db1 P a3ef849ee1e54c1b9e0fc8ba8a8f267c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6faa0ae89fb54a1eafe2f27278e53c89" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37745 } } peers { permanent_uuid: "24f99d81ad37473e9da0352cfc3101b0" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 41407 } } peers { permanent_uuid: "a3ef849ee1e54c1b9e0fc8ba8a8f267c" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 34897 } }
I20260811 06:29:29.684270 32338 leader_election.cc:288] T 2e82aa7807dc4aeda8bdbebd93818db1 P a3ef849ee1e54c1b9e0fc8ba8a8f267c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6faa0ae89fb54a1eafe2f27278e53c89 (127.28.239.194:37745), 24f99d81ad37473e9da0352cfc3101b0 (127.28.239.193:41407)
I20260811 06:29:29.690672 32316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:29.684728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.195:56251 (local address 127.28.239.194:37745)
0811 06:29:29.685178 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:29.685185 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:29.685211 (+ 26us) server_negotiation.cc:407] Connection header received
0811 06:29:29.685306 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:29.685314 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:29.685389 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:29.685502 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:29:29.686047 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:29.686880 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:29.688690 (+ 1810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:29.689675 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:29.689861 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:29.689977 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:29.690256 (+ 279us) server_negotiation.cc:299] Negotiation successful
0811 06:29:29.690400 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":240}
I20260811 06:29:29.690716 32130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:29.684755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.194:37745 (local address 127.28.239.195:56251)
0811 06:29:29.684996 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:29.685020 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 06:29:29.685168 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:29:29.685549 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:29:29.685555 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:29:29.685620 (+ 65us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:29:29.685863 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:29.685880 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:29.687061 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:29.687067 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:29.688501 (+ 1434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:29.688514 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:29.689844 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:29.689851 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:29.690063 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:29.690094 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 06:29:29.690211 (+ 117us) client_negotiation.cc:240] Negotiation successful
0811 06:29:29.690381 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":76}
I20260811 06:29:29.691460 32319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:29.684728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.195:58805 (local address 127.28.239.193:41407)
0811 06:29:29.684981 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:29.684991 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:29.685925 (+ 934us) server_negotiation.cc:407] Connection header received
0811 06:29:29.686121 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:29.686126 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:29.686198 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:29.686308 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:29:29.686826 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:29.687599 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:29.689304 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:29.690286 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:29.690490 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:29.690566 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:29.691003 (+ 437us) server_negotiation.cc:299] Negotiation successful
0811 06:29:29.691173 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260811 06:29:29.691586 32340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:29.684846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.193:41407 (local address 127.28.239.195:58805)
0811 06:29:29.685790 (+ 944us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:29.685808 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 06:29:29.685982 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:29:29.686319 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:29:29.686324 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:29:29.686347 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:29:29.686650 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:29.686660 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:29.687742 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:29.687754 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:29.689109 (+ 1355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:29.689121 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:29.690531 (+ 1410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:29.690537 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:29.690724 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:29.690779 (+ 55us) client_negotiation.cc:769] Sending connection context
0811 06:29:29.690930 (+ 151us) client_negotiation.cc:240] Negotiation successful
0811 06:29:29.691130 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":809,"thread_start_us":552,"threads_started":1}
I20260811 06:29:29.692075 32013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e82aa7807dc4aeda8bdbebd93818db1" candidate_uuid: "a3ef849ee1e54c1b9e0fc8ba8a8f267c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6faa0ae89fb54a1eafe2f27278e53c89" is_pre_election: true
I20260811 06:29:29.692719 32013 raft_consensus.cc:2454] T 2e82aa7807dc4aeda8bdbebd93818db1 P 6faa0ae89fb54a1eafe2f27278e53c89 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3ef849ee1e54c1b9e0fc8ba8a8f267c in term 0.
I20260811 06:29:29.692770 31788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e82aa7807dc4aeda8bdbebd93818db1" candidate_uuid: "a3ef849ee1e54c1b9e0fc8ba8a8f267c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24f99d81ad37473e9da0352cfc3101b0" is_pre_election: true
I20260811 06:29:29.693414 31788 raft_consensus.cc:2454] T 2e82aa7807dc4aeda8bdbebd93818db1 P 24f99d81ad37473e9da0352cfc3101b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3ef849ee1e54c1b9e0fc8ba8a8f267c in term 0.
I20260811 06:29:29.693970 32122 leader_election.cc:302] T 2e82aa7807dc4aeda8bdbebd93818db1 P a3ef849ee1e54c1b9e0fc8ba8a8f267c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6faa0ae89fb54a1eafe2f27278e53c89, a3ef849ee1e54c1b9e0fc8ba8a8f267c; no voters:
I20260811 06:29:29.694427 32338 raft_consensus.cc:2781] T 2e82aa7807dc4aeda8bdbebd93818db1 P a3ef849ee1e54c1b9e0fc8ba8a8f267c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:29:29.694587 32338 raft_consensus.cc:482] T 2e82aa7807dc4aeda8bdbebd93818db1 P a3ef849ee1e54c1b9e0fc8ba8a8f267c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:29:29.694690 32338 raft_consensus.cc:3037] T 2e82aa7807dc4aeda8bdbebd93818db1 P a3ef849ee1e54c1b9e0fc8ba8a8f267c [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:29:29.697013 32338 raft_consensus.cc:504] T 2e82aa7807dc4aeda8bdbebd93818db1 P a3ef849ee1e54c1b9e0fc8ba8a8f267c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6faa0ae89fb54a1eafe2f27278e53c89" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37745 } } peers { permanent_uuid: "24f99d81ad37473e9da0352cfc3101b0" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 41407 } } peers { permanent_uuid: "a3ef849ee1e54c1b9e0fc8ba8a8f267c" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 34897 } }
I20260811 06:29:29.697664 32338 leader_election.cc:288] T 2e82aa7807dc4aeda8bdbebd93818db1 P a3ef849ee1e54c1b9e0fc8ba8a8f267c [CANDIDATE]: Term 1 election: Requested vote from peers 6faa0ae89fb54a1eafe2f27278e53c89 (127.28.239.194:37745), 24f99d81ad37473e9da0352cfc3101b0 (127.28.239.193:41407)
I20260811 06:29:29.698175 31788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e82aa7807dc4aeda8bdbebd93818db1" candidate_uuid: "a3ef849ee1e54c1b9e0fc8ba8a8f267c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24f99d81ad37473e9da0352cfc3101b0"
I20260811 06:29:29.698237 32013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e82aa7807dc4aeda8bdbebd93818db1" candidate_uuid: "a3ef849ee1e54c1b9e0fc8ba8a8f267c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6faa0ae89fb54a1eafe2f27278e53c89"
I20260811 06:29:29.698385 31788 raft_consensus.cc:3037] T 2e82aa7807dc4aeda8bdbebd93818db1 P 24f99d81ad37473e9da0352cfc3101b0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:29:29.698482 32013 raft_consensus.cc:3037] T 2e82aa7807dc4aeda8bdbebd93818db1 P 6faa0ae89fb54a1eafe2f27278e53c89 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:29:29.700670 31788 raft_consensus.cc:2454] T 2e82aa7807dc4aeda8bdbebd93818db1 P 24f99d81ad37473e9da0352cfc3101b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3ef849ee1e54c1b9e0fc8ba8a8f267c in term 1.
I20260811 06:29:29.700855 32013 raft_consensus.cc:2454] T 2e82aa7807dc4aeda8bdbebd93818db1 P 6faa0ae89fb54a1eafe2f27278e53c89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3ef849ee1e54c1b9e0fc8ba8a8f267c in term 1.
I20260811 06:29:29.701108 32117 leader_election.cc:302] T 2e82aa7807dc4aeda8bdbebd93818db1 P a3ef849ee1e54c1b9e0fc8ba8a8f267c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24f99d81ad37473e9da0352cfc3101b0, a3ef849ee1e54c1b9e0fc8ba8a8f267c; no voters:
I20260811 06:29:29.701416 32338 raft_consensus.cc:2781] T 2e82aa7807dc4aeda8bdbebd93818db1 P a3ef849ee1e54c1b9e0fc8ba8a8f267c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:29:29.701916 32338 raft_consensus.cc:686] T 2e82aa7807dc4aeda8bdbebd93818db1 P a3ef849ee1e54c1b9e0fc8ba8a8f267c [term 1 LEADER]: Becoming Leader. State: Replica: a3ef849ee1e54c1b9e0fc8ba8a8f267c, State: Running, Role: LEADER
I20260811 06:29:29.702404 32338 consensus_queue.cc:237] T 2e82aa7807dc4aeda8bdbebd93818db1 P a3ef849ee1e54c1b9e0fc8ba8a8f267c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6faa0ae89fb54a1eafe2f27278e53c89" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37745 } } peers { permanent_uuid: "24f99d81ad37473e9da0352cfc3101b0" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 41407 } } peers { permanent_uuid: "a3ef849ee1e54c1b9e0fc8ba8a8f267c" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 34897 } }
I20260811 06:29:29.707372 31518 catalog_manager.cc:5495] T 2e82aa7807dc4aeda8bdbebd93818db1 P a3ef849ee1e54c1b9e0fc8ba8a8f267c reported cstate change: term changed from 0 to 1, leader changed from <none> to a3ef849ee1e54c1b9e0fc8ba8a8f267c (127.28.239.195). New cstate: current_term: 1 leader_uuid: "a3ef849ee1e54c1b9e0fc8ba8a8f267c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6faa0ae89fb54a1eafe2f27278e53c89" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24f99d81ad37473e9da0352cfc3101b0" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 41407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3ef849ee1e54c1b9e0fc8ba8a8f267c" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 34897 } health_report { overall_health: HEALTHY } } }
W20260811 06:29:29.708237 32297 tablet.cc:2369] T 2e82aa7807dc4aeda8bdbebd93818db1 P a3ef849ee1e54c1b9e0fc8ba8a8f267c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:29:29.748832 32321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:29.742819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50586 (local address 127.28.239.195:34897)
0811 06:29:29.743066 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:29.743074 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:29.743215 (+ 141us) server_negotiation.cc:407] Connection header received
0811 06:29:29.743623 (+ 408us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:29.743627 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:29.743734 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:29.743888 (+ 154us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:29:29.744290 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:29.745007 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:29.746423 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:29.746843 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:29.746987 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:29.747516 (+ 529us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:29:29.748111 (+ 595us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:29:29.748336 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:29.748388 (+ 52us) server_negotiation.cc:299] Negotiation successful
0811 06:29:29.748471 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 06:29:29.758491 31788 raft_consensus.cc:1261] T 2e82aa7807dc4aeda8bdbebd93818db1 P 24f99d81ad37473e9da0352cfc3101b0 [term 1 FOLLOWER]: Refusing update from remote peer a3ef849ee1e54c1b9e0fc8ba8a8f267c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:29:29.759694 32341 consensus_queue.cc:1035] T 2e82aa7807dc4aeda8bdbebd93818db1 P a3ef849ee1e54c1b9e0fc8ba8a8f267c [LEADER]: Connected to new peer: Peer: permanent_uuid: "24f99d81ad37473e9da0352cfc3101b0" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 41407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:29:29.760609 32013 raft_consensus.cc:1261] T 2e82aa7807dc4aeda8bdbebd93818db1 P 6faa0ae89fb54a1eafe2f27278e53c89 [term 1 FOLLOWER]: Refusing update from remote peer a3ef849ee1e54c1b9e0fc8ba8a8f267c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:29:29.761605 32341 consensus_queue.cc:1035] T 2e82aa7807dc4aeda8bdbebd93818db1 P a3ef849ee1e54c1b9e0fc8ba8a8f267c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6faa0ae89fb54a1eafe2f27278e53c89" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:29:29.773697 32346 mvcc.cc:205] Tried to move back new op lower bound from 7317216336918409216 to 7317216336705064960. Current Snapshot: MvccSnapshot[applied={T|T < 7317216336918409216}]
I20260811 06:29:29.780311 32350 mvcc.cc:205] Tried to move back new op lower bound from 7317216336918409216 to 7317216336705064960. Current Snapshot: MvccSnapshot[applied={T|T < 7317216336918409216}]
I20260811 06:29:29.797065 31516 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56750:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:29:29.797480 31516 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56750:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:29:29.802131 31516 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 66e064d761374e509bbcfe0045c9a9dd: Sending DeleteTablet for 3 replicas of tablet 2e82aa7807dc4aeda8bdbebd93818db1
I20260811 06:29:29.803544 31977 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e82aa7807dc4aeda8bdbebd93818db1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:29:29 UTC) from {username='kudu', principal='kudu/127.28.239.254@KRBTEST.COM'} at 127.0.0.1:58532
I20260811 06:29:29.803614 32193 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e82aa7807dc4aeda8bdbebd93818db1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:29:29 UTC) from {username='kudu', principal='kudu/127.28.239.254@KRBTEST.COM'} at 127.0.0.1:50570
I20260811 06:29:29.803728 31752 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e82aa7807dc4aeda8bdbebd93818db1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:29:29 UTC) from {username='kudu', principal='kudu/127.28.239.254@KRBTEST.COM'} at 127.0.0.1:59594
I20260811 06:29:29.809691 31516 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56750:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:29:29.810741 31516 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:29:29.814338 32365 tablet_replica.cc:330] T 2e82aa7807dc4aeda8bdbebd93818db1 P 24f99d81ad37473e9da0352cfc3101b0: stopping tablet replica
I20260811 06:29:29.815182 32365 raft_consensus.cc:2229] T 2e82aa7807dc4aeda8bdbebd93818db1 P 24f99d81ad37473e9da0352cfc3101b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:29:29.815930 32365 raft_consensus.cc:2258] T 2e82aa7807dc4aeda8bdbebd93818db1 P 24f99d81ad37473e9da0352cfc3101b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:29:29.821067 32364 tablet_replica.cc:330] T 2e82aa7807dc4aeda8bdbebd93818db1 P a3ef849ee1e54c1b9e0fc8ba8a8f267c: stopping tablet replica
I20260811 06:29:29.821818 32364 raft_consensus.cc:2229] T 2e82aa7807dc4aeda8bdbebd93818db1 P a3ef849ee1e54c1b9e0fc8ba8a8f267c [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:29:29.821889 32365 ts_tablet_manager.cc:1905] T 2e82aa7807dc4aeda8bdbebd93818db1 P 24f99d81ad37473e9da0352cfc3101b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:29:29.822413 31976 tablet_service.cc:1467] Processing CreateTablet for tablet 9d56fb22dc534469a9c6b2c2758873f0 (DEFAULT_TABLE table=test-table [id=b90a61635b064c73b81a587963383150]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:29:29.822902 32364 raft_consensus.cc:2258] T 2e82aa7807dc4aeda8bdbebd93818db1 P a3ef849ee1e54c1b9e0fc8ba8a8f267c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:29:29.823206 31976 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d56fb22dc534469a9c6b2c2758873f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:29:29.825448 32364 ts_tablet_manager.cc:1905] T 2e82aa7807dc4aeda8bdbebd93818db1 P a3ef849ee1e54c1b9e0fc8ba8a8f267c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:29:29.826149 32363 tablet_replica.cc:330] T 2e82aa7807dc4aeda8bdbebd93818db1 P 6faa0ae89fb54a1eafe2f27278e53c89: stopping tablet replica
I20260811 06:29:29.826244 31752 tablet_service.cc:1467] Processing CreateTablet for tablet 9d56fb22dc534469a9c6b2c2758873f0 (DEFAULT_TABLE table=test-table [id=b90a61635b064c73b81a587963383150]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:29:29.826727 31752 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d56fb22dc534469a9c6b2c2758873f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:29:29.831797 32327 tablet_bootstrap.cc:492] T 9d56fb22dc534469a9c6b2c2758873f0 P 6faa0ae89fb54a1eafe2f27278e53c89: Bootstrap starting.
I20260811 06:29:29.826342 32193 tablet_service.cc:1467] Processing CreateTablet for tablet 9d56fb22dc534469a9c6b2c2758873f0 (DEFAULT_TABLE table=test-table [id=b90a61635b064c73b81a587963383150]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:29:29.832353 32193 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d56fb22dc534469a9c6b2c2758873f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:29:29.833688 32322 tablet_bootstrap.cc:492] T 9d56fb22dc534469a9c6b2c2758873f0 P 24f99d81ad37473e9da0352cfc3101b0: Bootstrap starting.
I20260811 06:29:29.833737 32327 tablet_bootstrap.cc:654] T 9d56fb22dc534469a9c6b2c2758873f0 P 6faa0ae89fb54a1eafe2f27278e53c89: Neither blocks nor log segments found. Creating new log.
I20260811 06:29:29.835525 32322 tablet_bootstrap.cc:654] T 9d56fb22dc534469a9c6b2c2758873f0 P 24f99d81ad37473e9da0352cfc3101b0: Neither blocks nor log segments found. Creating new log.
I20260811 06:29:29.826701 32363 raft_consensus.cc:2229] T 2e82aa7807dc4aeda8bdbebd93818db1 P 6faa0ae89fb54a1eafe2f27278e53c89 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:29:29.841934 32365 ts_tablet_manager.cc:1918] T 2e82aa7807dc4aeda8bdbebd93818db1 P 24f99d81ad37473e9da0352cfc3101b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:29:29.842069 32363 raft_consensus.cc:2258] T 2e82aa7807dc4aeda8bdbebd93818db1 P 6faa0ae89fb54a1eafe2f27278e53c89 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:29:29.842207 32365 log.cc:1196] T 2e82aa7807dc4aeda8bdbebd93818db1 P 24f99d81ad37473e9da0352cfc3101b0: Deleting WAL directory at /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/minicluster-data/ts-0/wal/wals/2e82aa7807dc4aeda8bdbebd93818db1
I20260811 06:29:29.842971 32365 ts_tablet_manager.cc:1939] T 2e82aa7807dc4aeda8bdbebd93818db1 P 24f99d81ad37473e9da0352cfc3101b0: Deleting consensus metadata
I20260811 06:29:29.844713 32364 ts_tablet_manager.cc:1918] T 2e82aa7807dc4aeda8bdbebd93818db1 P a3ef849ee1e54c1b9e0fc8ba8a8f267c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:29:29.845212 31496 catalog_manager.cc:4841] TS 24f99d81ad37473e9da0352cfc3101b0 (127.28.239.193:41407): tablet 2e82aa7807dc4aeda8bdbebd93818db1 (table test-table [id=c329d285d6784d55b2fdcc9323e0065a]) successfully deleted
I20260811 06:29:29.845474 32330 tablet_bootstrap.cc:492] T 9d56fb22dc534469a9c6b2c2758873f0 P a3ef849ee1e54c1b9e0fc8ba8a8f267c: Bootstrap starting.
I20260811 06:29:29.848192 32327 tablet_bootstrap.cc:492] T 9d56fb22dc534469a9c6b2c2758873f0 P 6faa0ae89fb54a1eafe2f27278e53c89: No bootstrap required, opened a new log
I20260811 06:29:29.848313 32330 tablet_bootstrap.cc:654] T 9d56fb22dc534469a9c6b2c2758873f0 P a3ef849ee1e54c1b9e0fc8ba8a8f267c: Neither blocks nor log segments found. Creating new log.
I20260811 06:29:29.848412 32322 tablet_bootstrap.cc:492] T 9d56fb22dc534469a9c6b2c2758873f0 P 24f99d81ad37473e9da0352cfc3101b0: No bootstrap required, opened a new log
I20260811 06:29:29.848532 32327 ts_tablet_manager.cc:1397] T 9d56fb22dc534469a9c6b2c2758873f0 P 6faa0ae89fb54a1eafe2f27278e53c89: Time spent bootstrapping tablet: real 0.017s user 0.006s sys 0.005s
I20260811 06:29:29.848592 32322 ts_tablet_manager.cc:1397] T 9d56fb22dc534469a9c6b2c2758873f0 P 24f99d81ad37473e9da0352cfc3101b0: Time spent bootstrapping tablet: real 0.015s user 0.010s sys 0.002s
I20260811 06:29:29.849510 32327 raft_consensus.cc:348] T 9d56fb22dc534469a9c6b2c2758873f0 P 6faa0ae89fb54a1eafe2f27278e53c89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24f99d81ad37473e9da0352cfc3101b0" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 41407 } } peers { permanent_uuid: "6faa0ae89fb54a1eafe2f27278e53c89" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37745 } } peers { permanent_uuid: "a3ef849ee1e54c1b9e0fc8ba8a8f267c" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 34897 } }
I20260811 06:29:29.849601 32322 raft_consensus.cc:348] T 9d56fb22dc534469a9c6b2c2758873f0 P 24f99d81ad37473e9da0352cfc3101b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24f99d81ad37473e9da0352cfc3101b0" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 41407 } } peers { permanent_uuid: "6faa0ae89fb54a1eafe2f27278e53c89" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37745 } } peers { permanent_uuid: "a3ef849ee1e54c1b9e0fc8ba8a8f267c" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 34897 } }
I20260811 06:29:29.849864 32327 raft_consensus.cc:374] T 9d56fb22dc534469a9c6b2c2758873f0 P 6faa0ae89fb54a1eafe2f27278e53c89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:29:29.849948 32322 raft_consensus.cc:374] T 9d56fb22dc534469a9c6b2c2758873f0 P 24f99d81ad37473e9da0352cfc3101b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:29:29.850069 32322 raft_consensus.cc:729] T 9d56fb22dc534469a9c6b2c2758873f0 P 24f99d81ad37473e9da0352cfc3101b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24f99d81ad37473e9da0352cfc3101b0, State: Initialized, Role: FOLLOWER
I20260811 06:29:29.850108 32327 raft_consensus.cc:729] T 9d56fb22dc534469a9c6b2c2758873f0 P 6faa0ae89fb54a1eafe2f27278e53c89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6faa0ae89fb54a1eafe2f27278e53c89, State: Initialized, Role: FOLLOWER
I20260811 06:29:29.850569 32327 consensus_queue.cc:260] T 9d56fb22dc534469a9c6b2c2758873f0 P 6faa0ae89fb54a1eafe2f27278e53c89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24f99d81ad37473e9da0352cfc3101b0" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 41407 } } peers { permanent_uuid: "6faa0ae89fb54a1eafe2f27278e53c89" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37745 } } peers { permanent_uuid: "a3ef849ee1e54c1b9e0fc8ba8a8f267c" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 34897 } }
I20260811 06:29:29.850545 32322 consensus_queue.cc:260] T 9d56fb22dc534469a9c6b2c2758873f0 P 24f99d81ad37473e9da0352cfc3101b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24f99d81ad37473e9da0352cfc3101b0" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 41407 } } peers { permanent_uuid: "6faa0ae89fb54a1eafe2f27278e53c89" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37745 } } peers { permanent_uuid: "a3ef849ee1e54c1b9e0fc8ba8a8f267c" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 34897 } }
I20260811 06:29:29.853169 32327 ts_tablet_manager.cc:1428] T 9d56fb22dc534469a9c6b2c2758873f0 P 6faa0ae89fb54a1eafe2f27278e53c89: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260811 06:29:29.854111 32364 log.cc:1196] T 2e82aa7807dc4aeda8bdbebd93818db1 P a3ef849ee1e54c1b9e0fc8ba8a8f267c: Deleting WAL directory at /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/minicluster-data/ts-2/wal/wals/2e82aa7807dc4aeda8bdbebd93818db1
I20260811 06:29:29.856257 32363 ts_tablet_manager.cc:1905] T 2e82aa7807dc4aeda8bdbebd93818db1 P 6faa0ae89fb54a1eafe2f27278e53c89: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:29:29.856925 32364 ts_tablet_manager.cc:1939] T 2e82aa7807dc4aeda8bdbebd93818db1 P a3ef849ee1e54c1b9e0fc8ba8a8f267c: Deleting consensus metadata
I20260811 06:29:29.858508 32322 ts_tablet_manager.cc:1428] T 9d56fb22dc534469a9c6b2c2758873f0 P 24f99d81ad37473e9da0352cfc3101b0: Time spent starting tablet: real 0.010s user 0.000s sys 0.003s
I20260811 06:29:29.859884 31499 catalog_manager.cc:4841] TS a3ef849ee1e54c1b9e0fc8ba8a8f267c (127.28.239.195:34897): tablet 2e82aa7807dc4aeda8bdbebd93818db1 (table test-table [id=c329d285d6784d55b2fdcc9323e0065a]) successfully deleted
W20260811 06:29:29.858959 32089 tablet.cc:2369] T 9d56fb22dc534469a9c6b2c2758873f0 P 6faa0ae89fb54a1eafe2f27278e53c89: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:29:29.863471 32330 tablet_bootstrap.cc:492] T 9d56fb22dc534469a9c6b2c2758873f0 P a3ef849ee1e54c1b9e0fc8ba8a8f267c: No bootstrap required, opened a new log
I20260811 06:29:29.863651 32330 ts_tablet_manager.cc:1397] T 9d56fb22dc534469a9c6b2c2758873f0 P a3ef849ee1e54c1b9e0fc8ba8a8f267c: Time spent bootstrapping tablet: real 0.018s user 0.010s sys 0.005s
I20260811 06:29:29.864612 32330 raft_consensus.cc:348] T 9d56fb22dc534469a9c6b2c2758873f0 P a3ef849ee1e54c1b9e0fc8ba8a8f267c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24f99d81ad37473e9da0352cfc3101b0" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 41407 } } peers { permanent_uuid: "6faa0ae89fb54a1eafe2f27278e53c89" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37745 } } peers { permanent_uuid: "a3ef849ee1e54c1b9e0fc8ba8a8f267c" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 34897 } }
I20260811 06:29:29.864887 32330 raft_consensus.cc:374] T 9d56fb22dc534469a9c6b2c2758873f0 P a3ef849ee1e54c1b9e0fc8ba8a8f267c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:29:29.865005 32330 raft_consensus.cc:729] T 9d56fb22dc534469a9c6b2c2758873f0 P a3ef849ee1e54c1b9e0fc8ba8a8f267c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3ef849ee1e54c1b9e0fc8ba8a8f267c, State: Initialized, Role: FOLLOWER
I20260811 06:29:29.865350 32330 consensus_queue.cc:260] T 9d56fb22dc534469a9c6b2c2758873f0 P a3ef849ee1e54c1b9e0fc8ba8a8f267c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24f99d81ad37473e9da0352cfc3101b0" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 41407 } } peers { permanent_uuid: "6faa0ae89fb54a1eafe2f27278e53c89" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37745 } } peers { permanent_uuid: "a3ef849ee1e54c1b9e0fc8ba8a8f267c" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 34897 } }
I20260811 06:29:29.869079 32363 ts_tablet_manager.cc:1918] T 2e82aa7807dc4aeda8bdbebd93818db1 P 6faa0ae89fb54a1eafe2f27278e53c89: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:29:29.869299 32363 log.cc:1196] T 2e82aa7807dc4aeda8bdbebd93818db1 P 6faa0ae89fb54a1eafe2f27278e53c89: Deleting WAL directory at /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/minicluster-data/ts-1/wal/wals/2e82aa7807dc4aeda8bdbebd93818db1
I20260811 06:29:29.869391 32330 ts_tablet_manager.cc:1428] T 9d56fb22dc534469a9c6b2c2758873f0 P a3ef849ee1e54c1b9e0fc8ba8a8f267c: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
I20260811 06:29:29.869987 32363 ts_tablet_manager.cc:1939] T 2e82aa7807dc4aeda8bdbebd93818db1 P 6faa0ae89fb54a1eafe2f27278e53c89: Deleting consensus metadata
I20260811 06:29:29.871974 31500 catalog_manager.cc:4841] TS 6faa0ae89fb54a1eafe2f27278e53c89 (127.28.239.194:37745): tablet 2e82aa7807dc4aeda8bdbebd93818db1 (table test-table [id=c329d285d6784d55b2fdcc9323e0065a]) successfully deleted
I20260811 06:29:30.073421 32335 raft_consensus.cc:482] T 9d56fb22dc534469a9c6b2c2758873f0 P 6faa0ae89fb54a1eafe2f27278e53c89 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:29:30.073707 32335 raft_consensus.cc:504] T 9d56fb22dc534469a9c6b2c2758873f0 P 6faa0ae89fb54a1eafe2f27278e53c89 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24f99d81ad37473e9da0352cfc3101b0" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 41407 } } peers { permanent_uuid: "6faa0ae89fb54a1eafe2f27278e53c89" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37745 } } peers { permanent_uuid: "a3ef849ee1e54c1b9e0fc8ba8a8f267c" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 34897 } }
W20260811 06:29:30.074038 31866 tablet.cc:2369] T 9d56fb22dc534469a9c6b2c2758873f0 P 24f99d81ad37473e9da0352cfc3101b0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:29:30.075562 32335 leader_election.cc:288] T 9d56fb22dc534469a9c6b2c2758873f0 P 6faa0ae89fb54a1eafe2f27278e53c89 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24f99d81ad37473e9da0352cfc3101b0 (127.28.239.193:41407), a3ef849ee1e54c1b9e0fc8ba8a8f267c (127.28.239.195:34897)
I20260811 06:29:30.085004 32319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:30.076097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.194:35547 (local address 127.28.239.193:41407)
0811 06:29:30.076528 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:30.076535 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:30.077227 (+ 692us) server_negotiation.cc:407] Connection header received
0811 06:29:30.077576 (+ 349us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:30.077583 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:30.077706 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:30.077814 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:29:30.078511 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:30.079667 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:30.082467 (+ 2800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:30.083848 (+ 1381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:30.084036 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:30.084119 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:30.084539 (+ 420us) server_negotiation.cc:299] Negotiation successful
0811 06:29:30.084700 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278}
I20260811 06:29:30.085327 32321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:30.076693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.194:53485 (local address 127.28.239.195:34897)
0811 06:29:30.076895 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:30.076903 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:30.077853 (+ 950us) server_negotiation.cc:407] Connection header received
0811 06:29:30.078225 (+ 372us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:30.078235 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:30.078332 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:30.078403 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:29:30.079369 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:30.080576 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:30.082649 (+ 2073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:30.084002 (+ 1353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:30.084112 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:30.084190 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:30.084845 (+ 655us) server_negotiation.cc:299] Negotiation successful
0811 06:29:30.084990 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 06:29:30.085844 32373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:30.075973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.193:41407 (local address 127.28.239.194:35547)
0811 06:29:30.077083 (+ 1110us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:30.077111 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0811 06:29:30.077396 (+ 285us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:29:30.077836 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:29:30.077841 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:29:30.077870 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:29:30.078358 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:30.078373 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:30.079798 (+ 1425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:30.079804 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:30.082293 (+ 2489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:30.082309 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:30.084100 (+ 1791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:30.084106 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:30.084320 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:30.084362 (+ 42us) client_negotiation.cc:769] Sending connection context
0811 06:29:30.085213 (+ 851us) client_negotiation.cc:240] Negotiation successful
0811 06:29:30.085366 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":925,"mutex_wait_us":1,"thread_start_us":1304,"threads_started":1}
I20260811 06:29:30.086190 32372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:30.076049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.195:34897 (local address 127.28.239.194:53485)
0811 06:29:30.077704 (+ 1655us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:30.077764 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 06:29:30.078027 (+ 263us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:29:30.078705 (+ 678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:29:30.078709 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:29:30.078737 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:29:30.079153 (+ 416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:30.079165 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:30.080705 (+ 1540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:30.080712 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:30.082440 (+ 1728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:30.082457 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:30.084423 (+ 1966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:30.084432 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:30.084623 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:30.084660 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 06:29:30.085592 (+ 932us) client_negotiation.cc:240] Negotiation successful
0811 06:29:30.085738 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1499,"thread_start_us":835,"threads_started":1}
I20260811 06:29:30.086692 31788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d56fb22dc534469a9c6b2c2758873f0" candidate_uuid: "6faa0ae89fb54a1eafe2f27278e53c89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24f99d81ad37473e9da0352cfc3101b0" is_pre_election: true
I20260811 06:29:30.086987 31788 raft_consensus.cc:2454] T 9d56fb22dc534469a9c6b2c2758873f0 P 24f99d81ad37473e9da0352cfc3101b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6faa0ae89fb54a1eafe2f27278e53c89 in term 0.
I20260811 06:29:30.087064 32230 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d56fb22dc534469a9c6b2c2758873f0" candidate_uuid: "6faa0ae89fb54a1eafe2f27278e53c89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3ef849ee1e54c1b9e0fc8ba8a8f267c" is_pre_election: true
I20260811 06:29:30.087566 32230 raft_consensus.cc:2454] T 9d56fb22dc534469a9c6b2c2758873f0 P a3ef849ee1e54c1b9e0fc8ba8a8f267c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6faa0ae89fb54a1eafe2f27278e53c89 in term 0.
I20260811 06:29:30.088253 31890 leader_election.cc:302] T 9d56fb22dc534469a9c6b2c2758873f0 P 6faa0ae89fb54a1eafe2f27278e53c89 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 24f99d81ad37473e9da0352cfc3101b0, 6faa0ae89fb54a1eafe2f27278e53c89, a3ef849ee1e54c1b9e0fc8ba8a8f267c; no voters:
I20260811 06:29:30.088733 32335 raft_consensus.cc:2781] T 9d56fb22dc534469a9c6b2c2758873f0 P 6faa0ae89fb54a1eafe2f27278e53c89 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:29:30.088877 32335 raft_consensus.cc:482] T 9d56fb22dc534469a9c6b2c2758873f0 P 6faa0ae89fb54a1eafe2f27278e53c89 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:29:30.089042 32335 raft_consensus.cc:3037] T 9d56fb22dc534469a9c6b2c2758873f0 P 6faa0ae89fb54a1eafe2f27278e53c89 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:29:30.091794 32335 raft_consensus.cc:504] T 9d56fb22dc534469a9c6b2c2758873f0 P 6faa0ae89fb54a1eafe2f27278e53c89 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24f99d81ad37473e9da0352cfc3101b0" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 41407 } } peers { permanent_uuid: "6faa0ae89fb54a1eafe2f27278e53c89" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37745 } } peers { permanent_uuid: "a3ef849ee1e54c1b9e0fc8ba8a8f267c" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 34897 } }
I20260811 06:29:30.092430 32335 leader_election.cc:288] T 9d56fb22dc534469a9c6b2c2758873f0 P 6faa0ae89fb54a1eafe2f27278e53c89 [CANDIDATE]: Term 1 election: Requested vote from peers 24f99d81ad37473e9da0352cfc3101b0 (127.28.239.193:41407), a3ef849ee1e54c1b9e0fc8ba8a8f267c (127.28.239.195:34897)
I20260811 06:29:30.092949 32230 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d56fb22dc534469a9c6b2c2758873f0" candidate_uuid: "6faa0ae89fb54a1eafe2f27278e53c89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3ef849ee1e54c1b9e0fc8ba8a8f267c"
I20260811 06:29:30.092975 31788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d56fb22dc534469a9c6b2c2758873f0" candidate_uuid: "6faa0ae89fb54a1eafe2f27278e53c89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24f99d81ad37473e9da0352cfc3101b0"
I20260811 06:29:30.093163 32230 raft_consensus.cc:3037] T 9d56fb22dc534469a9c6b2c2758873f0 P a3ef849ee1e54c1b9e0fc8ba8a8f267c [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:29:30.093184 31788 raft_consensus.cc:3037] T 9d56fb22dc534469a9c6b2c2758873f0 P 24f99d81ad37473e9da0352cfc3101b0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:29:30.095583 32230 raft_consensus.cc:2454] T 9d56fb22dc534469a9c6b2c2758873f0 P a3ef849ee1e54c1b9e0fc8ba8a8f267c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6faa0ae89fb54a1eafe2f27278e53c89 in term 1.
I20260811 06:29:30.096042 31894 leader_election.cc:302] T 9d56fb22dc534469a9c6b2c2758873f0 P 6faa0ae89fb54a1eafe2f27278e53c89 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6faa0ae89fb54a1eafe2f27278e53c89, a3ef849ee1e54c1b9e0fc8ba8a8f267c; no voters:
I20260811 06:29:30.096381 32335 raft_consensus.cc:2781] T 9d56fb22dc534469a9c6b2c2758873f0 P 6faa0ae89fb54a1eafe2f27278e53c89 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:29:30.096364 31788 raft_consensus.cc:2454] T 9d56fb22dc534469a9c6b2c2758873f0 P 24f99d81ad37473e9da0352cfc3101b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6faa0ae89fb54a1eafe2f27278e53c89 in term 1.
I20260811 06:29:30.096958 32335 raft_consensus.cc:686] T 9d56fb22dc534469a9c6b2c2758873f0 P 6faa0ae89fb54a1eafe2f27278e53c89 [term 1 LEADER]: Becoming Leader. State: Replica: 6faa0ae89fb54a1eafe2f27278e53c89, State: Running, Role: LEADER
I20260811 06:29:30.097404 32335 consensus_queue.cc:237] T 9d56fb22dc534469a9c6b2c2758873f0 P 6faa0ae89fb54a1eafe2f27278e53c89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24f99d81ad37473e9da0352cfc3101b0" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 41407 } } peers { permanent_uuid: "6faa0ae89fb54a1eafe2f27278e53c89" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37745 } } peers { permanent_uuid: "a3ef849ee1e54c1b9e0fc8ba8a8f267c" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 34897 } }
I20260811 06:29:30.101190 31516 catalog_manager.cc:5495] T 9d56fb22dc534469a9c6b2c2758873f0 P 6faa0ae89fb54a1eafe2f27278e53c89 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6faa0ae89fb54a1eafe2f27278e53c89 (127.28.239.194). New cstate: current_term: 1 leader_uuid: "6faa0ae89fb54a1eafe2f27278e53c89" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24f99d81ad37473e9da0352cfc3101b0" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 41407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6faa0ae89fb54a1eafe2f27278e53c89" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37745 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a3ef849ee1e54c1b9e0fc8ba8a8f267c" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 34897 } health_report { overall_health: UNKNOWN } } }
Aug 11 06:29:30 dist-test-slave-gkw7 krb5kdc[31458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429767, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.254@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:29:30.183285 32386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:30.168238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56762 (local address 127.28.239.254:43627)
0811 06:29:30.169212 (+ 974us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:30.169217 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:30.169240 (+ 23us) server_negotiation.cc:407] Connection header received
0811 06:29:30.169281 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:30.169285 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:30.169338 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:30.169451 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:30.170634 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:30.171413 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:30.172715 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:30.173137 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:30.173270 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:30.177083 (+ 3813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:29:30.177088 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:29:30.177092 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:29:30.177121 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:29:30.180304 (+ 3183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:30.181001 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:30.181006 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:30.181010 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:30.181147 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:30.181555 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:30.181561 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:30.181564 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:30.181909 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:29:30.182052 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:30.182491 (+ 439us) server_negotiation.cc:299] Negotiation successful
0811 06:29:30.182861 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":164,"server-negotiator.queue_time_us":830,"thread_start_us":683,"threads_started":1}
I20260811 06:29:30.183244 32385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:30.168157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:43627 (local address 127.0.0.1:56762)
0811 06:29:30.168864 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:30.168891 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 06:29:30.169082 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:29:30.169488 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:29:30.169495 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:29:30.170239 (+ 744us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:29:30.170460 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:30.170470 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:30.171561 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:30.171568 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:30.172569 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:30.172582 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:30.173310 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:30.173317 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:30.173479 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:30.174315 (+ 836us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:29:30.174333 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:29:30.176886 (+ 2553us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:29:30.180436 (+ 3550us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:29:30.180448 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:29:30.180456 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:29:30.180878 (+ 422us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:29:30.181255 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:29:30.181269 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:29:30.181272 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:29:30.181419 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:29:30.182044 (+ 625us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:29:30.182052 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:29:30.182232 (+ 180us) client_negotiation.cc:769] Sending connection context
0811 06:29:30.182411 (+ 179us) client_negotiation.cc:240] Negotiation successful
0811 06:29:30.182737 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":538,"mutex_wait_us":3,"thread_start_us":381,"threads_started":1}
I20260811 06:29:30.189981 31516 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.28.239.254@KRBTEST.COM'} at 127.0.0.1:56762:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 06:29:30.191478 31516 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:29:30.199365 32193 tablet_service.cc:1467] Processing CreateTablet for tablet b4ffdef5c32d49cdb8558579f1546363 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=fcc4d6ce06894cf29311c4450fe82aba]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 06:29:30.199337 31752 tablet_service.cc:1467] Processing CreateTablet for tablet b4ffdef5c32d49cdb8558579f1546363 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=fcc4d6ce06894cf29311c4450fe82aba]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 06:29:30.199451 31976 tablet_service.cc:1467] Processing CreateTablet for tablet b4ffdef5c32d49cdb8558579f1546363 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=fcc4d6ce06894cf29311c4450fe82aba]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 06:29:30.199925 32193 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4ffdef5c32d49cdb8558579f1546363. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:29:30.199937 31752 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4ffdef5c32d49cdb8558579f1546363. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:29:30.200062 31976 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4ffdef5c32d49cdb8558579f1546363. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:29:30.208004 32330 tablet_bootstrap.cc:492] T b4ffdef5c32d49cdb8558579f1546363 P a3ef849ee1e54c1b9e0fc8ba8a8f267c: Bootstrap starting.
I20260811 06:29:30.209794 32322 tablet_bootstrap.cc:492] T b4ffdef5c32d49cdb8558579f1546363 P 24f99d81ad37473e9da0352cfc3101b0: Bootstrap starting.
I20260811 06:29:30.209992 32330 tablet_bootstrap.cc:654] T b4ffdef5c32d49cdb8558579f1546363 P a3ef849ee1e54c1b9e0fc8ba8a8f267c: Neither blocks nor log segments found. Creating new log.
I20260811 06:29:30.211400 32322 tablet_bootstrap.cc:654] T b4ffdef5c32d49cdb8558579f1546363 P 24f99d81ad37473e9da0352cfc3101b0: Neither blocks nor log segments found. Creating new log.
I20260811 06:29:30.212811 32327 tablet_bootstrap.cc:492] T b4ffdef5c32d49cdb8558579f1546363 P 6faa0ae89fb54a1eafe2f27278e53c89: Bootstrap starting.
I20260811 06:29:30.214510 32327 tablet_bootstrap.cc:654] T b4ffdef5c32d49cdb8558579f1546363 P 6faa0ae89fb54a1eafe2f27278e53c89: Neither blocks nor log segments found. Creating new log.
I20260811 06:29:30.215085 32322 tablet_bootstrap.cc:492] T b4ffdef5c32d49cdb8558579f1546363 P 24f99d81ad37473e9da0352cfc3101b0: No bootstrap required, opened a new log
I20260811 06:29:30.215332 32322 ts_tablet_manager.cc:1397] T b4ffdef5c32d49cdb8558579f1546363 P 24f99d81ad37473e9da0352cfc3101b0: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.001s
I20260811 06:29:30.217425 32322 raft_consensus.cc:348] T b4ffdef5c32d49cdb8558579f1546363 P 24f99d81ad37473e9da0352cfc3101b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24f99d81ad37473e9da0352cfc3101b0" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 41407 } } peers { permanent_uuid: "6faa0ae89fb54a1eafe2f27278e53c89" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37745 } } peers { permanent_uuid: "a3ef849ee1e54c1b9e0fc8ba8a8f267c" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 34897 } }
I20260811 06:29:30.218024 32322 raft_consensus.cc:374] T b4ffdef5c32d49cdb8558579f1546363 P 24f99d81ad37473e9da0352cfc3101b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:29:30.218180 32322 raft_consensus.cc:729] T b4ffdef5c32d49cdb8558579f1546363 P 24f99d81ad37473e9da0352cfc3101b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24f99d81ad37473e9da0352cfc3101b0, State: Initialized, Role: FOLLOWER
I20260811 06:29:30.218621 32322 consensus_queue.cc:260] T b4ffdef5c32d49cdb8558579f1546363 P 24f99d81ad37473e9da0352cfc3101b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24f99d81ad37473e9da0352cfc3101b0" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 41407 } } peers { permanent_uuid: "6faa0ae89fb54a1eafe2f27278e53c89" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37745 } } peers { permanent_uuid: "a3ef849ee1e54c1b9e0fc8ba8a8f267c" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 34897 } }
I20260811 06:29:30.219465 32334 tablet_replica.cc:439] T b4ffdef5c32d49cdb8558579f1546363 P 24f99d81ad37473e9da0352cfc3101b0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24f99d81ad37473e9da0352cfc3101b0" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 41407 } } peers { permanent_uuid: "6faa0ae89fb54a1eafe2f27278e53c89" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37745 } } peers { permanent_uuid: "a3ef849ee1e54c1b9e0fc8ba8a8f267c" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 34897 } } }
I20260811 06:29:30.220009 32334 tablet_replica.cc:442] T b4ffdef5c32d49cdb8558579f1546363 P 24f99d81ad37473e9da0352cfc3101b0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 06:29:30.221513 32322 ts_tablet_manager.cc:1428] T b4ffdef5c32d49cdb8558579f1546363 P 24f99d81ad37473e9da0352cfc3101b0: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
I20260811 06:29:30.224388 32330 tablet_bootstrap.cc:492] T b4ffdef5c32d49cdb8558579f1546363 P a3ef849ee1e54c1b9e0fc8ba8a8f267c: No bootstrap required, opened a new log
I20260811 06:29:30.224588 32330 ts_tablet_manager.cc:1397] T b4ffdef5c32d49cdb8558579f1546363 P a3ef849ee1e54c1b9e0fc8ba8a8f267c: Time spent bootstrapping tablet: real 0.017s user 0.005s sys 0.010s
I20260811 06:29:30.225452 32330 raft_consensus.cc:348] T b4ffdef5c32d49cdb8558579f1546363 P a3ef849ee1e54c1b9e0fc8ba8a8f267c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24f99d81ad37473e9da0352cfc3101b0" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 41407 } } peers { permanent_uuid: "6faa0ae89fb54a1eafe2f27278e53c89" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37745 } } peers { permanent_uuid: "a3ef849ee1e54c1b9e0fc8ba8a8f267c" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 34897 } }
I20260811 06:29:30.225757 32330 raft_consensus.cc:374] T b4ffdef5c32d49cdb8558579f1546363 P a3ef849ee1e54c1b9e0fc8ba8a8f267c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:29:30.225839 32330 raft_consensus.cc:729] T b4ffdef5c32d49cdb8558579f1546363 P a3ef849ee1e54c1b9e0fc8ba8a8f267c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3ef849ee1e54c1b9e0fc8ba8a8f267c, State: Initialized, Role: FOLLOWER
I20260811 06:29:30.226155 32330 consensus_queue.cc:260] T b4ffdef5c32d49cdb8558579f1546363 P a3ef849ee1e54c1b9e0fc8ba8a8f267c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24f99d81ad37473e9da0352cfc3101b0" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 41407 } } peers { permanent_uuid: "6faa0ae89fb54a1eafe2f27278e53c89" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37745 } } peers { permanent_uuid: "a3ef849ee1e54c1b9e0fc8ba8a8f267c" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 34897 } }
I20260811 06:29:30.226622 32338 tablet_replica.cc:439] T b4ffdef5c32d49cdb8558579f1546363 P a3ef849ee1e54c1b9e0fc8ba8a8f267c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24f99d81ad37473e9da0352cfc3101b0" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 41407 } } peers { permanent_uuid: "6faa0ae89fb54a1eafe2f27278e53c89" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37745 } } peers { permanent_uuid: "a3ef849ee1e54c1b9e0fc8ba8a8f267c" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 34897 } } }
I20260811 06:29:30.226971 32338 tablet_replica.cc:442] T b4ffdef5c32d49cdb8558579f1546363 P a3ef849ee1e54c1b9e0fc8ba8a8f267c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 06:29:30.228282 32330 ts_tablet_manager.cc:1428] T b4ffdef5c32d49cdb8558579f1546363 P a3ef849ee1e54c1b9e0fc8ba8a8f267c: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260811 06:29:30.229910 32327 tablet_bootstrap.cc:492] T b4ffdef5c32d49cdb8558579f1546363 P 6faa0ae89fb54a1eafe2f27278e53c89: No bootstrap required, opened a new log
I20260811 06:29:30.230104 32327 ts_tablet_manager.cc:1397] T b4ffdef5c32d49cdb8558579f1546363 P 6faa0ae89fb54a1eafe2f27278e53c89: Time spent bootstrapping tablet: real 0.017s user 0.003s sys 0.012s
I20260811 06:29:30.231000 32327 raft_consensus.cc:348] T b4ffdef5c32d49cdb8558579f1546363 P 6faa0ae89fb54a1eafe2f27278e53c89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24f99d81ad37473e9da0352cfc3101b0" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 41407 } } peers { permanent_uuid: "6faa0ae89fb54a1eafe2f27278e53c89" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37745 } } peers { permanent_uuid: "a3ef849ee1e54c1b9e0fc8ba8a8f267c" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 34897 } }
I20260811 06:29:30.231243 32327 raft_consensus.cc:374] T b4ffdef5c32d49cdb8558579f1546363 P 6faa0ae89fb54a1eafe2f27278e53c89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:29:30.231325 32327 raft_consensus.cc:729] T b4ffdef5c32d49cdb8558579f1546363 P 6faa0ae89fb54a1eafe2f27278e53c89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6faa0ae89fb54a1eafe2f27278e53c89, State: Initialized, Role: FOLLOWER
I20260811 06:29:30.231624 32327 consensus_queue.cc:260] T b4ffdef5c32d49cdb8558579f1546363 P 6faa0ae89fb54a1eafe2f27278e53c89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24f99d81ad37473e9da0352cfc3101b0" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 41407 } } peers { permanent_uuid: "6faa0ae89fb54a1eafe2f27278e53c89" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37745 } } peers { permanent_uuid: "a3ef849ee1e54c1b9e0fc8ba8a8f267c" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 34897 } }
I20260811 06:29:30.232234 32327 ts_tablet_manager.cc:1428] T b4ffdef5c32d49cdb8558579f1546363 P 6faa0ae89fb54a1eafe2f27278e53c89: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 06:29:30.232077 32335 tablet_replica.cc:439] T b4ffdef5c32d49cdb8558579f1546363 P 6faa0ae89fb54a1eafe2f27278e53c89: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24f99d81ad37473e9da0352cfc3101b0" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 41407 } } peers { permanent_uuid: "6faa0ae89fb54a1eafe2f27278e53c89" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37745 } } peers { permanent_uuid: "a3ef849ee1e54c1b9e0fc8ba8a8f267c" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 34897 } } }
I20260811 06:29:30.232406 32335 tablet_replica.cc:442] T b4ffdef5c32d49cdb8558579f1546363 P 6faa0ae89fb54a1eafe2f27278e53c89: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 06:29:30.343436 32335 raft_consensus.cc:482] T b4ffdef5c32d49cdb8558579f1546363 P 6faa0ae89fb54a1eafe2f27278e53c89 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:29:30.343642 32335 raft_consensus.cc:504] T b4ffdef5c32d49cdb8558579f1546363 P 6faa0ae89fb54a1eafe2f27278e53c89 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24f99d81ad37473e9da0352cfc3101b0" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 41407 } } peers { permanent_uuid: "6faa0ae89fb54a1eafe2f27278e53c89" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37745 } } peers { permanent_uuid: "a3ef849ee1e54c1b9e0fc8ba8a8f267c" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 34897 } }
I20260811 06:29:30.344262 32335 leader_election.cc:288] T b4ffdef5c32d49cdb8558579f1546363 P 6faa0ae89fb54a1eafe2f27278e53c89 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24f99d81ad37473e9da0352cfc3101b0 (127.28.239.193:41407), a3ef849ee1e54c1b9e0fc8ba8a8f267c (127.28.239.195:34897)
I20260811 06:29:30.344842 31788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4ffdef5c32d49cdb8558579f1546363" candidate_uuid: "6faa0ae89fb54a1eafe2f27278e53c89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24f99d81ad37473e9da0352cfc3101b0" is_pre_election: true
I20260811 06:29:30.344856 32230 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4ffdef5c32d49cdb8558579f1546363" candidate_uuid: "6faa0ae89fb54a1eafe2f27278e53c89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3ef849ee1e54c1b9e0fc8ba8a8f267c" is_pre_election: true
I20260811 06:29:30.345104 31788 raft_consensus.cc:2454] T b4ffdef5c32d49cdb8558579f1546363 P 24f99d81ad37473e9da0352cfc3101b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6faa0ae89fb54a1eafe2f27278e53c89 in term 0.
I20260811 06:29:30.345131 32230 raft_consensus.cc:2454] T b4ffdef5c32d49cdb8558579f1546363 P a3ef849ee1e54c1b9e0fc8ba8a8f267c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6faa0ae89fb54a1eafe2f27278e53c89 in term 0.
I20260811 06:29:30.345594 31890 leader_election.cc:302] T b4ffdef5c32d49cdb8558579f1546363 P 6faa0ae89fb54a1eafe2f27278e53c89 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24f99d81ad37473e9da0352cfc3101b0, 6faa0ae89fb54a1eafe2f27278e53c89; no voters:
I20260811 06:29:30.345916 32335 raft_consensus.cc:2781] T b4ffdef5c32d49cdb8558579f1546363 P 6faa0ae89fb54a1eafe2f27278e53c89 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:29:30.346025 32335 raft_consensus.cc:482] T b4ffdef5c32d49cdb8558579f1546363 P 6faa0ae89fb54a1eafe2f27278e53c89 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:29:30.346210 32335 raft_consensus.cc:3037] T b4ffdef5c32d49cdb8558579f1546363 P 6faa0ae89fb54a1eafe2f27278e53c89 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:29:30.348531 32335 raft_consensus.cc:504] T b4ffdef5c32d49cdb8558579f1546363 P 6faa0ae89fb54a1eafe2f27278e53c89 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24f99d81ad37473e9da0352cfc3101b0" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 41407 } } peers { permanent_uuid: "6faa0ae89fb54a1eafe2f27278e53c89" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37745 } } peers { permanent_uuid: "a3ef849ee1e54c1b9e0fc8ba8a8f267c" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 34897 } }
I20260811 06:29:30.349182 32335 leader_election.cc:288] T b4ffdef5c32d49cdb8558579f1546363 P 6faa0ae89fb54a1eafe2f27278e53c89 [CANDIDATE]: Term 1 election: Requested vote from peers 24f99d81ad37473e9da0352cfc3101b0 (127.28.239.193:41407), a3ef849ee1e54c1b9e0fc8ba8a8f267c (127.28.239.195:34897)
I20260811 06:29:30.349644 31788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4ffdef5c32d49cdb8558579f1546363" candidate_uuid: "6faa0ae89fb54a1eafe2f27278e53c89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24f99d81ad37473e9da0352cfc3101b0"
I20260811 06:29:30.349745 32230 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4ffdef5c32d49cdb8558579f1546363" candidate_uuid: "6faa0ae89fb54a1eafe2f27278e53c89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3ef849ee1e54c1b9e0fc8ba8a8f267c"
I20260811 06:29:30.349879 31788 raft_consensus.cc:3037] T b4ffdef5c32d49cdb8558579f1546363 P 24f99d81ad37473e9da0352cfc3101b0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:29:30.350003 32230 raft_consensus.cc:3037] T b4ffdef5c32d49cdb8558579f1546363 P a3ef849ee1e54c1b9e0fc8ba8a8f267c [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:29:30.352262 32230 raft_consensus.cc:2454] T b4ffdef5c32d49cdb8558579f1546363 P a3ef849ee1e54c1b9e0fc8ba8a8f267c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6faa0ae89fb54a1eafe2f27278e53c89 in term 1.
I20260811 06:29:30.352535 31788 raft_consensus.cc:2454] T b4ffdef5c32d49cdb8558579f1546363 P 24f99d81ad37473e9da0352cfc3101b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6faa0ae89fb54a1eafe2f27278e53c89 in term 1.
I20260811 06:29:30.352712 31894 leader_election.cc:302] T b4ffdef5c32d49cdb8558579f1546363 P 6faa0ae89fb54a1eafe2f27278e53c89 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6faa0ae89fb54a1eafe2f27278e53c89, a3ef849ee1e54c1b9e0fc8ba8a8f267c; no voters:
I20260811 06:29:30.353052 32335 raft_consensus.cc:2781] T b4ffdef5c32d49cdb8558579f1546363 P 6faa0ae89fb54a1eafe2f27278e53c89 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:29:30.353269 32335 raft_consensus.cc:686] T b4ffdef5c32d49cdb8558579f1546363 P 6faa0ae89fb54a1eafe2f27278e53c89 [term 1 LEADER]: Becoming Leader. State: Replica: 6faa0ae89fb54a1eafe2f27278e53c89, State: Running, Role: LEADER
I20260811 06:29:30.353631 32335 consensus_queue.cc:237] T b4ffdef5c32d49cdb8558579f1546363 P 6faa0ae89fb54a1eafe2f27278e53c89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24f99d81ad37473e9da0352cfc3101b0" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 41407 } } peers { permanent_uuid: "6faa0ae89fb54a1eafe2f27278e53c89" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37745 } } peers { permanent_uuid: "a3ef849ee1e54c1b9e0fc8ba8a8f267c" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 34897 } }
I20260811 06:29:30.355726 32374 tablet_replica.cc:439] T b4ffdef5c32d49cdb8558579f1546363 P 6faa0ae89fb54a1eafe2f27278e53c89: TxnStatusTablet state changed. Reason: New leader 6faa0ae89fb54a1eafe2f27278e53c89. Latest consensus state: current_term: 1 leader_uuid: "6faa0ae89fb54a1eafe2f27278e53c89" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24f99d81ad37473e9da0352cfc3101b0" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 41407 } } peers { permanent_uuid: "6faa0ae89fb54a1eafe2f27278e53c89" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37745 } } peers { permanent_uuid: "a3ef849ee1e54c1b9e0fc8ba8a8f267c" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 34897 } } }
I20260811 06:29:30.356107 32374 tablet_replica.cc:442] T b4ffdef5c32d49cdb8558579f1546363 P 6faa0ae89fb54a1eafe2f27278e53c89: This TxnStatusTablet replica's current role is: LEADER
I20260811 06:29:30.357013 32393 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 06:29:30.357352 32393 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 06:29:30.357383 31516 catalog_manager.cc:5495] T b4ffdef5c32d49cdb8558579f1546363 P 6faa0ae89fb54a1eafe2f27278e53c89 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6faa0ae89fb54a1eafe2f27278e53c89 (127.28.239.194). New cstate: current_term: 1 leader_uuid: "6faa0ae89fb54a1eafe2f27278e53c89" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24f99d81ad37473e9da0352cfc3101b0" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 41407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6faa0ae89fb54a1eafe2f27278e53c89" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37745 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a3ef849ee1e54c1b9e0fc8ba8a8f267c" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 34897 } health_report { overall_health: UNKNOWN } } }
I20260811 06:29:30.383322 32385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:30.375891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.194:37745 (local address 127.0.0.1:58534)
0811 06:29:30.376168 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:30.376197 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0811 06:29:30.376412 (+ 215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:29:30.377243 (+ 831us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:29:30.377247 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:29:30.377256 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 06:29:30.377433 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:30.377441 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:30.379108 (+ 1667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:30.379111 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:30.380370 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:30.380384 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:30.381395 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:30.381399 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:30.381617 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:30.381682 (+ 65us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 06:29:30.382779 (+ 1097us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 06:29:30.382796 (+ 17us) client_negotiation.cc:769] Sending connection context
0811 06:29:30.382855 (+ 59us) client_negotiation.cc:240] Negotiation successful
0811 06:29:30.382975 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
I20260811 06:29:30.383387 32402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:30.375827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58534 (local address 127.28.239.194:37745)
0811 06:29:30.376672 (+ 845us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:30.376679 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:30.376716 (+ 37us) server_negotiation.cc:407] Connection header received
0811 06:29:30.376819 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:30.376825 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:30.376984 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:30.377194 (+ 210us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:29:30.377666 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:30.378985 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:30.380514 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:30.381229 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:30.381365 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:30.381886 (+ 521us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:29:30.382662 (+ 776us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:29:30.382821 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:30.382881 (+ 60us) server_negotiation.cc:299] Negotiation successful
0811 06:29:30.382968 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":695,"thread_start_us":565,"threads_started":1}
I20260811 06:29:30.388475 31788 raft_consensus.cc:1261] T b4ffdef5c32d49cdb8558579f1546363 P 24f99d81ad37473e9da0352cfc3101b0 [term 1 FOLLOWER]: Refusing update from remote peer 6faa0ae89fb54a1eafe2f27278e53c89: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:29:30.388797 32230 raft_consensus.cc:1261] T b4ffdef5c32d49cdb8558579f1546363 P a3ef849ee1e54c1b9e0fc8ba8a8f267c [term 1 FOLLOWER]: Refusing update from remote peer 6faa0ae89fb54a1eafe2f27278e53c89: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:29:30.389343 32335 consensus_queue.cc:1035] T b4ffdef5c32d49cdb8558579f1546363 P 6faa0ae89fb54a1eafe2f27278e53c89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24f99d81ad37473e9da0352cfc3101b0" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 41407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:29:30.389792 32374 consensus_queue.cc:1035] T b4ffdef5c32d49cdb8558579f1546363 P 6faa0ae89fb54a1eafe2f27278e53c89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3ef849ee1e54c1b9e0fc8ba8a8f267c" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 34897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:29:30.399326 32338 tablet_replica.cc:439] T b4ffdef5c32d49cdb8558579f1546363 P a3ef849ee1e54c1b9e0fc8ba8a8f267c: TxnStatusTablet state changed. Reason: New leader 6faa0ae89fb54a1eafe2f27278e53c89. Latest consensus state: current_term: 1 leader_uuid: "6faa0ae89fb54a1eafe2f27278e53c89" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24f99d81ad37473e9da0352cfc3101b0" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 41407 } } peers { permanent_uuid: "6faa0ae89fb54a1eafe2f27278e53c89" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37745 } } peers { permanent_uuid: "a3ef849ee1e54c1b9e0fc8ba8a8f267c" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 34897 } } }
I20260811 06:29:30.399664 32338 tablet_replica.cc:442] T b4ffdef5c32d49cdb8558579f1546363 P a3ef849ee1e54c1b9e0fc8ba8a8f267c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 06:29:30.402791 32374 tablet_replica.cc:439] T b4ffdef5c32d49cdb8558579f1546363 P 6faa0ae89fb54a1eafe2f27278e53c89: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6faa0ae89fb54a1eafe2f27278e53c89" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24f99d81ad37473e9da0352cfc3101b0" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 41407 } } peers { permanent_uuid: "6faa0ae89fb54a1eafe2f27278e53c89" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37745 } } peers { permanent_uuid: "a3ef849ee1e54c1b9e0fc8ba8a8f267c" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 34897 } } }
I20260811 06:29:30.403064 32374 tablet_replica.cc:442] T b4ffdef5c32d49cdb8558579f1546363 P 6faa0ae89fb54a1eafe2f27278e53c89: This TxnStatusTablet replica's current role is: LEADER
I20260811 06:29:30.406078 32338 tablet_replica.cc:439] T b4ffdef5c32d49cdb8558579f1546363 P a3ef849ee1e54c1b9e0fc8ba8a8f267c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6faa0ae89fb54a1eafe2f27278e53c89" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24f99d81ad37473e9da0352cfc3101b0" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 41407 } } peers { permanent_uuid: "6faa0ae89fb54a1eafe2f27278e53c89" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37745 } } peers { permanent_uuid: "a3ef849ee1e54c1b9e0fc8ba8a8f267c" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 34897 } } }
I20260811 06:29:30.406368 32338 tablet_replica.cc:442] T b4ffdef5c32d49cdb8558579f1546363 P a3ef849ee1e54c1b9e0fc8ba8a8f267c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 06:29:30.410602 32334 tablet_replica.cc:439] T b4ffdef5c32d49cdb8558579f1546363 P 24f99d81ad37473e9da0352cfc3101b0: TxnStatusTablet state changed. Reason: New leader 6faa0ae89fb54a1eafe2f27278e53c89. Latest consensus state: current_term: 1 leader_uuid: "6faa0ae89fb54a1eafe2f27278e53c89" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24f99d81ad37473e9da0352cfc3101b0" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 41407 } } peers { permanent_uuid: "6faa0ae89fb54a1eafe2f27278e53c89" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37745 } } peers { permanent_uuid: "a3ef849ee1e54c1b9e0fc8ba8a8f267c" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 34897 } } }
I20260811 06:29:30.410956 32334 tablet_replica.cc:442] T b4ffdef5c32d49cdb8558579f1546363 P 24f99d81ad37473e9da0352cfc3101b0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 06:29:30.411823 32335 tablet_replica.cc:439] T b4ffdef5c32d49cdb8558579f1546363 P 6faa0ae89fb54a1eafe2f27278e53c89: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6faa0ae89fb54a1eafe2f27278e53c89" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24f99d81ad37473e9da0352cfc3101b0" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 41407 } } peers { permanent_uuid: "6faa0ae89fb54a1eafe2f27278e53c89" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37745 } } peers { permanent_uuid: "a3ef849ee1e54c1b9e0fc8ba8a8f267c" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 34897 } } }
I20260811 06:29:30.412200 32335 tablet_replica.cc:442] T b4ffdef5c32d49cdb8558579f1546363 P 6faa0ae89fb54a1eafe2f27278e53c89: This TxnStatusTablet replica's current role is: LEADER
I20260811 06:29:30.424089 32334 tablet_replica.cc:439] T b4ffdef5c32d49cdb8558579f1546363 P 24f99d81ad37473e9da0352cfc3101b0: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6faa0ae89fb54a1eafe2f27278e53c89" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24f99d81ad37473e9da0352cfc3101b0" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 41407 } } peers { permanent_uuid: "6faa0ae89fb54a1eafe2f27278e53c89" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37745 } } peers { permanent_uuid: "a3ef849ee1e54c1b9e0fc8ba8a8f267c" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 34897 } } }
I20260811 06:29:30.424553 32334 tablet_replica.cc:442] T b4ffdef5c32d49cdb8558579f1546363 P 24f99d81ad37473e9da0352cfc3101b0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 06:29:30.431039 32402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:30.422664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58542 (local address 127.28.239.194:37745)
0811 06:29:30.422862 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:30.422869 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:30.425955 (+ 3086us) server_negotiation.cc:407] Connection header received
0811 06:29:30.426071 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:30.426076 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:30.426166 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:30.426300 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:29:30.426903 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:30.427673 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:30.429130 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:30.429671 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:30.429809 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:30.430255 (+ 446us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:29:30.430366 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:29:30.430479 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:30.430639 (+ 160us) server_negotiation.cc:299] Negotiation successful
0811 06:29:30.430730 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 06:29:30.450426 32402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:30.443981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.194:52525 (local address 127.28.239.194:37745)
0811 06:29:30.444175 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:30.444179 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:30.445055 (+ 876us) server_negotiation.cc:407] Connection header received
0811 06:29:30.445301 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:30.445306 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:30.445391 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:30.445506 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:29:30.446046 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:30.446827 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:30.448404 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:30.448954 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:30.449088 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:30.449617 (+ 529us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:29:30.449727 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:29:30.449826 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:30.449976 (+ 150us) server_negotiation.cc:299] Negotiation successful
0811 06:29:30.450046 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 06:29:30.450542 32423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:30.444205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.194:37745 (local address 127.28.239.194:52525)
0811 06:29:30.444944 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:30.444961 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 06:29:30.445146 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:29:30.445511 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:29:30.445515 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:29:30.445525 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 06:29:30.445877 (+ 352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:30.445887 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:30.446953 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:30.446960 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:30.448181 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:30.448193 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:30.449178 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:30.449183 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:30.449327 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:30.449385 (+ 58us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 06:29:30.449831 (+ 446us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 06:29:30.449844 (+ 13us) client_negotiation.cc:769] Sending connection context
0811 06:29:30.449942 (+ 98us) client_negotiation.cc:240] Negotiation successful
0811 06:29:30.450074 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":596,"thread_start_us":392,"threads_started":1}
I20260811 06:29:30.462188 32230 raft_consensus.cc:1261] T 9d56fb22dc534469a9c6b2c2758873f0 P a3ef849ee1e54c1b9e0fc8ba8a8f267c [term 1 FOLLOWER]: Refusing update from remote peer 6faa0ae89fb54a1eafe2f27278e53c89: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:29:30.462211 31788 raft_consensus.cc:1261] T 9d56fb22dc534469a9c6b2c2758873f0 P 24f99d81ad37473e9da0352cfc3101b0 [term 1 FOLLOWER]: Refusing update from remote peer 6faa0ae89fb54a1eafe2f27278e53c89: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:29:30.462917 32416 consensus_queue.cc:1035] T 9d56fb22dc534469a9c6b2c2758873f0 P 6faa0ae89fb54a1eafe2f27278e53c89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3ef849ee1e54c1b9e0fc8ba8a8f267c" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 34897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:29:30.463238 32374 consensus_queue.cc:1035] T 9d56fb22dc534469a9c6b2c2758873f0 P 6faa0ae89fb54a1eafe2f27278e53c89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24f99d81ad37473e9da0352cfc3101b0" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 41407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:29:30.551169 31516 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56750:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:29:30.551441 31516 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56750:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:29:30.554842 31516 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 66e064d761374e509bbcfe0045c9a9dd: Sending DeleteTablet for 3 replicas of tablet 9d56fb22dc534469a9c6b2c2758873f0
I20260811 06:29:30.555791 31752 tablet_service.cc:1514] Processing DeleteTablet for tablet 9d56fb22dc534469a9c6b2c2758873f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:29:30 UTC) from {username='kudu', principal='kudu/127.28.239.254@KRBTEST.COM'} at 127.0.0.1:59594
I20260811 06:29:30.556156 32193 tablet_service.cc:1514] Processing DeleteTablet for tablet 9d56fb22dc534469a9c6b2c2758873f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:29:30 UTC) from {username='kudu', principal='kudu/127.28.239.254@KRBTEST.COM'} at 127.0.0.1:50570
I20260811 06:29:30.556361 31976 tablet_service.cc:1514] Processing DeleteTablet for tablet 9d56fb22dc534469a9c6b2c2758873f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:29:30 UTC) from {username='kudu', principal='kudu/127.28.239.254@KRBTEST.COM'} at 127.0.0.1:58532
I20260811 06:29:30.562790 32436 tablet_replica.cc:330] T 9d56fb22dc534469a9c6b2c2758873f0 P 6faa0ae89fb54a1eafe2f27278e53c89: stopping tablet replica
I20260811 06:29:30.563105 32436 raft_consensus.cc:2229] T 9d56fb22dc534469a9c6b2c2758873f0 P 6faa0ae89fb54a1eafe2f27278e53c89 [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:29:30.563927 32436 raft_consensus.cc:2258] T 9d56fb22dc534469a9c6b2c2758873f0 P 6faa0ae89fb54a1eafe2f27278e53c89 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:29:30.568270 32436 ts_tablet_manager.cc:1905] T 9d56fb22dc534469a9c6b2c2758873f0 P 6faa0ae89fb54a1eafe2f27278e53c89: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:29:30.568554 32432 tablet_replica.cc:330] T 9d56fb22dc534469a9c6b2c2758873f0 P a3ef849ee1e54c1b9e0fc8ba8a8f267c: stopping tablet replica
I20260811 06:29:30.568858 32432 raft_consensus.cc:2229] T 9d56fb22dc534469a9c6b2c2758873f0 P a3ef849ee1e54c1b9e0fc8ba8a8f267c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:29:30.569265 32432 raft_consensus.cc:2258] T 9d56fb22dc534469a9c6b2c2758873f0 P a3ef849ee1e54c1b9e0fc8ba8a8f267c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:29:30.571034 32427 tablet_replica.cc:330] T 9d56fb22dc534469a9c6b2c2758873f0 P 24f99d81ad37473e9da0352cfc3101b0: stopping tablet replica
I20260811 06:29:30.571396 32427 raft_consensus.cc:2229] T 9d56fb22dc534469a9c6b2c2758873f0 P 24f99d81ad37473e9da0352cfc3101b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:29:30.571791 32427 raft_consensus.cc:2258] T 9d56fb22dc534469a9c6b2c2758873f0 P 24f99d81ad37473e9da0352cfc3101b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:29:30.573261 32432 ts_tablet_manager.cc:1905] T 9d56fb22dc534469a9c6b2c2758873f0 P a3ef849ee1e54c1b9e0fc8ba8a8f267c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:29:30.573429 32427 ts_tablet_manager.cc:1905] T 9d56fb22dc534469a9c6b2c2758873f0 P 24f99d81ad37473e9da0352cfc3101b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:29:30.577874 32436 ts_tablet_manager.cc:1918] T 9d56fb22dc534469a9c6b2c2758873f0 P 6faa0ae89fb54a1eafe2f27278e53c89: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 06:29:30.578058 32436 log.cc:1196] T 9d56fb22dc534469a9c6b2c2758873f0 P 6faa0ae89fb54a1eafe2f27278e53c89: Deleting WAL directory at /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/minicluster-data/ts-1/wal/wals/9d56fb22dc534469a9c6b2c2758873f0
I20260811 06:29:30.578714 32436 ts_tablet_manager.cc:1939] T 9d56fb22dc534469a9c6b2c2758873f0 P 6faa0ae89fb54a1eafe2f27278e53c89: Deleting consensus metadata
I20260811 06:29:30.580261 31500 catalog_manager.cc:4841] TS 6faa0ae89fb54a1eafe2f27278e53c89 (127.28.239.194:37745): tablet 9d56fb22dc534469a9c6b2c2758873f0 (table test-table [id=b90a61635b064c73b81a587963383150]) successfully deleted
I20260811 06:29:30.582094 32427 ts_tablet_manager.cc:1918] T 9d56fb22dc534469a9c6b2c2758873f0 P 24f99d81ad37473e9da0352cfc3101b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 06:29:30.582386 32427 log.cc:1196] T 9d56fb22dc534469a9c6b2c2758873f0 P 24f99d81ad37473e9da0352cfc3101b0: Deleting WAL directory at /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/minicluster-data/ts-0/wal/wals/9d56fb22dc534469a9c6b2c2758873f0
I20260811 06:29:30.582984 32427 ts_tablet_manager.cc:1939] T 9d56fb22dc534469a9c6b2c2758873f0 P 24f99d81ad37473e9da0352cfc3101b0: Deleting consensus metadata
I20260811 06:29:30.584210 31496 catalog_manager.cc:4841] TS 24f99d81ad37473e9da0352cfc3101b0 (127.28.239.193:41407): tablet 9d56fb22dc534469a9c6b2c2758873f0 (table test-table [id=b90a61635b064c73b81a587963383150]) successfully deleted
I20260811 06:29:30.585235 32432 ts_tablet_manager.cc:1918] T 9d56fb22dc534469a9c6b2c2758873f0 P a3ef849ee1e54c1b9e0fc8ba8a8f267c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 06:29:30.585398 32432 log.cc:1196] T 9d56fb22dc534469a9c6b2c2758873f0 P a3ef849ee1e54c1b9e0fc8ba8a8f267c: Deleting WAL directory at /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786429755927508-29631-0/minicluster-data/ts-2/wal/wals/9d56fb22dc534469a9c6b2c2758873f0
I20260811 06:29:30.585971 32432 ts_tablet_manager.cc:1939] T 9d56fb22dc534469a9c6b2c2758873f0 P a3ef849ee1e54c1b9e0fc8ba8a8f267c: Deleting consensus metadata
I20260811 06:29:30.587129 31499 catalog_manager.cc:4841] TS a3ef849ee1e54c1b9e0fc8ba8a8f267c (127.28.239.195:34897): tablet 9d56fb22dc534469a9c6b2c2758873f0 (table test-table [id=b90a61635b064c73b81a587963383150]) successfully deleted
Aug 11 06:29:30 dist-test-slave-gkw7 krb5kdc[31458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429769, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.28.239.193@KRBTEST.COM
I20260811 06:29:30.597237 32319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:30.566833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59608 (local address 127.28.239.193:41407)
0811 06:29:30.567054 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:30.567063 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:30.579249 (+ 12186us) server_negotiation.cc:407] Connection header received
0811 06:29:30.579562 (+ 313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:30.579568 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:30.579677 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:30.579775 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:30.582205 (+ 2430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:30.583403 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:30.585176 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:30.585892 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:30.586050 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:30.590957 (+ 4907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:29:30.590980 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:29:30.590996 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:29:30.591030 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:29:30.594282 (+ 3252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:30.595046 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:30.595054 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:30.595056 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:30.595204 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:30.595626 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:30.595630 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:30.595632 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:30.595991 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:29:30.596349 (+ 358us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:30.596654 (+ 305us) server_negotiation.cc:299] Negotiation successful
0811 06:29:30.596886 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
W20260811 06:29:30.598263 31857 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59608
I20260811 06:29:30.617208 32386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:30.601448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56764 (local address 127.28.239.254:43627)
0811 06:29:30.601708 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:30.601715 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:30.601962 (+ 247us) server_negotiation.cc:407] Connection header received
0811 06:29:30.602170 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:30.602174 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:30.602241 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:30.602324 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:30.604303 (+ 1979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:30.605061 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:30.607171 (+ 2110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:30.607609 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:30.607712 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:30.610987 (+ 3275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:29:30.610992 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:29:30.610995 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:29:30.611032 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:29:30.614060 (+ 3028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:30.614960 (+ 900us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:30.614964 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:30.614967 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:30.615089 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:30.615497 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:30.615500 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:30.615502 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:30.615792 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:29:30.615974 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:30.616518 (+ 544us) server_negotiation.cc:299] Negotiation successful
0811 06:29:30.616756 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120}
W20260811 06:29:30.618242 31516 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56764
I20260811 06:29:30.621853 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 31637
I20260811 06:29:30.640707 32319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:30.623209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59610 (local address 127.28.239.193:41407)
0811 06:29:30.623448 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:30.623455 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:30.624574 (+ 1119us) server_negotiation.cc:407] Connection header received
0811 06:29:30.624755 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:30.624761 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:30.624849 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:30.624967 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:30.626569 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:30.627722 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:30.629091 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:30.629810 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:30.629943 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:30.633435 (+ 3492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:29:30.633442 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:29:30.633447 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:29:30.633493 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:29:30.637226 (+ 3733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:30.638208 (+ 982us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:30.638214 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:30.638217 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:30.638371 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:30.638866 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:30.638871 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:30.638874 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:30.639224 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:29:30.639500 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:30.640032 (+ 532us) server_negotiation.cc:299] Negotiation successful
0811 06:29:30.640320 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
W20260811 06:29:30.641345 31857 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59610
W20260811 06:29:30.642091 29631 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 31637: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 06:29:30.642305 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 31637
I20260811 06:29:30.676487 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 31869
Aug 11 06:29:30 dist-test-slave-gkw7 krb5kdc[31458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429769, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.28.239.194@KRBTEST.COM
I20260811 06:29:30.696444 32402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:30.677630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58554 (local address 127.28.239.194:37745)
0811 06:29:30.677913 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:30.677921 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:30.678082 (+ 161us) server_negotiation.cc:407] Connection header received
0811 06:29:30.678326 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:30.678332 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:30.678459 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:30.678609 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:30.680368 (+ 1759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:30.681617 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:30.683004 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:30.683561 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:30.683717 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:30.688239 (+ 4522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:29:30.688253 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:29:30.688273 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:29:30.688315 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:29:30.692326 (+ 4011us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:30.693282 (+ 956us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:30.693292 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:30.693295 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:30.693484 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:30.694303 (+ 819us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:30.694316 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:30.694321 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:30.694853 (+ 532us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:29:30.695337 (+ 484us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:30.695698 (+ 361us) server_negotiation.cc:299] Negotiation successful
0811 06:29:30.695992 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110}
W20260811 06:29:30.697314 32079 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58554
W20260811 06:29:30.698182 29631 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 31869: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 06:29:30.698416 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 31869
I20260811 06:29:30.731766 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 32093
Aug 11 06:29:30 dist-test-slave-gkw7 krb5kdc[31458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429769, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.28.239.195@KRBTEST.COM
I20260811 06:29:30.747789 32446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:30.732650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50602 (local address 127.28.239.195:34897)
0811 06:29:30.733180 (+ 530us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:30.733186 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:30.733203 (+ 17us) server_negotiation.cc:407] Connection header received
0811 06:29:30.733269 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:30.733272 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:30.733332 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:30.733429 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:30.734817 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:30.735526 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:30.736797 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:30.737223 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:30.737333 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:30.742058 (+ 4725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:29:30.742076 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:29:30.742095 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:29:30.742145 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:29:30.745027 (+ 2882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:30.745917 (+ 890us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:30.745924 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:30.745927 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:30.746077 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:30.746411 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:30.746414 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:30.746416 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:30.746775 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:29:30.747018 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:30.747280 (+ 262us) server_negotiation.cc:299] Negotiation successful
0811 06:29:30.747515 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":435,"thread_start_us":304,"threads_started":1}
W20260811 06:29:30.748610 32288 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50602
W20260811 06:29:30.749320 29631 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 32093: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 06:29:30.749488 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 32093
I20260811 06:29:30.783490 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 31476
I20260811 06:29:30.784740 31592 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:29:30.906198 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 31476
2026-08-11T06:29:30Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (5276 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:29:31 dist-test-slave-gkw7 krb5kdc[32451](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:29:31 dist-test-slave-gkw7 krb5kdc[32451](info): set up 2 sockets
Aug 11 06:29:31 dist-test-slave-gkw7 krb5kdc[32451](info): commencing operation
krb5kdc: starting...
W20260811 06:29:32.241863 29631 mini_kdc.cc:121] Time spent starting KDC: real 1.278s user 0.004s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/krb5kdc/test-user.keytab.
Aug 11 06:29:32 dist-test-slave-gkw7 krb5kdc[32451](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429772, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:29:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:29:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.239.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.239.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.239.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab.
Entry for principal kudu/127.28.239.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab.
I20260811 06:29:32.465296 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:46267
--webserver_interface=127.28.239.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab
--principal=kudu/127.28.239.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:42809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.239.254:46267
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:29:32.813205 32467 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:29:32.813637 32467 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:29:32.813833 32467 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:29:32.821488 32467 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:29:32.821733 32467 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 06:29:32.821866 32467 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:29:32.821972 32467 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:29:32.822086 32467 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:29:32.822214 32467 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:29:32.832899 32467 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:42809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.239.254:46267
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab
--principal=kudu/127.28.239.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:46267
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.239.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:29:32.834434 32467 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:29:32.835906 32467 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:29:32.848719 32483 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:29:32.848819 32478 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:29:32.848850 32476 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:29:32.849732 32467 server_base.cc:1034] running on GCE node
I20260811 06:29:32.850571 32467 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:29:32.851998 32467 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:29:32.853261 32467 hybrid_clock.cc:648] HybridClock initialized: now 1786429772853203 us; error 39 us; skew 500 ppm
I20260811 06:29:32.853752 32467 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:29:32 dist-test-slave-gkw7 krb5kdc[32451](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429772, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:29:32.857079 32467 init.cc:377] Logged in from keytab as kudu/127.28.239.254@KRBTEST.COM (short username kudu)
I20260811 06:29:32.858671 32467 webserver.cc:460] Webserver started at http://127.28.239.254:44019/ using document root <none> and password file <none>
I20260811 06:29:32.859413 32467 fs_manager.cc:359] Metadata directory not provided
I20260811 06:29:32.859548 32467 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:29:32.859884 32467 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:29:32.863019 32467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/minicluster-data/master-0/data/instance:
uuid: "15e16a76dba440648ce59c6d9f4b7a8c"
format_stamp: "Formatted at 2026-08-11 06:29:32 on dist-test-slave-gkw7"
server_key: "d25524685eff0bb1e8ac40d7607ae37b"
server_key_iv: "2dd5039a0c0b5340951aa8e871a9073b"
server_key_version: "encryptionkey@0"
I20260811 06:29:32.863818 32467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/minicluster-data/master-0/wal/instance:
uuid: "15e16a76dba440648ce59c6d9f4b7a8c"
format_stamp: "Formatted at 2026-08-11 06:29:32 on dist-test-slave-gkw7"
server_key: "d25524685eff0bb1e8ac40d7607ae37b"
server_key_iv: "2dd5039a0c0b5340951aa8e871a9073b"
server_key_version: "encryptionkey@0"
I20260811 06:29:32.869266 32467 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 06:29:32.872892 32496 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:29:32.874295 32467 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:29:32.874488 32467 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/minicluster-data/master-0/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/minicluster-data/master-0/wal
uuid: "15e16a76dba440648ce59c6d9f4b7a8c"
format_stamp: "Formatted at 2026-08-11 06:29:32 on dist-test-slave-gkw7"
server_key: "d25524685eff0bb1e8ac40d7607ae37b"
server_key_iv: "2dd5039a0c0b5340951aa8e871a9073b"
server_key_version: "encryptionkey@0"
I20260811 06:29:32.874763 32467 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:29:32.897348 32467 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:29:32.902201 32467 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:29:32.902475 32467 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:29:32.926884 32467 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.254:46267
I20260811 06:29:32.926890 32602 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.254:46267 every 8 connection(s)
I20260811 06:29:32.928577 32467 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
I20260811 06:29:32.932636 32603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:29:32.936293 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 32467
I20260811 06:29:32.936554 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/minicluster-data/master-0/wal/instance
I20260811 06:29:32.937317 29631 external_mini_cluster.cc:1428] Setting key f87f0e4274d5219bc2866afd4a50c951
I20260811 06:29:32.946772 32603 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15e16a76dba440648ce59c6d9f4b7a8c: Bootstrap starting.
I20260811 06:29:32.950917 32603 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 15e16a76dba440648ce59c6d9f4b7a8c: Neither blocks nor log segments found. Creating new log.
I20260811 06:29:32.952383 32603 log.cc:824] T 00000000000000000000000000000000 P 15e16a76dba440648ce59c6d9f4b7a8c: Log is configured to *not* fsync() on all Append() calls
Aug 11 06:29:32 dist-test-slave-gkw7 krb5kdc[32451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429772, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:29:32.955628 32603 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15e16a76dba440648ce59c6d9f4b7a8c: No bootstrap required, opened a new log
I20260811 06:29:32.963637 32610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:32.939666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50266 (local address 127.28.239.254:46267)
0811 06:29:32.940584 (+ 918us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:32.940596 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:32.940646 (+ 50us) server_negotiation.cc:407] Connection header received
0811 06:29:32.942494 (+ 1848us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:32.942527 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:32.943000 (+ 473us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:32.943387 (+ 387us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:32.945234 (+ 1847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:32.946734 (+ 1500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:32.948555 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:32.949354 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:32.949509 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:32.954908 (+ 5399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:29:32.954918 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:29:32.954935 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:29:32.954974 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:29:32.958553 (+ 3579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:32.959361 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:32.959369 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:32.959376 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:32.959613 (+ 237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:32.960219 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:32.960224 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:32.960227 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:32.961000 (+ 773us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:29:32.961362 (+ 362us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:32.961937 (+ 575us) server_negotiation.cc:299] Negotiation successful
0811 06:29:32.962475 (+ 538us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":677,"thread_start_us":449,"threads_started":1}
I20260811 06:29:32.964150 32603 raft_consensus.cc:348] T 00000000000000000000000000000000 P 15e16a76dba440648ce59c6d9f4b7a8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15e16a76dba440648ce59c6d9f4b7a8c" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 46267 } }
I20260811 06:29:32.964556 32603 raft_consensus.cc:374] T 00000000000000000000000000000000 P 15e16a76dba440648ce59c6d9f4b7a8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:29:32.964673 32603 raft_consensus.cc:729] T 00000000000000000000000000000000 P 15e16a76dba440648ce59c6d9f4b7a8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15e16a76dba440648ce59c6d9f4b7a8c, State: Initialized, Role: FOLLOWER
I20260811 06:29:32.965387 32603 consensus_queue.cc:260] T 00000000000000000000000000000000 P 15e16a76dba440648ce59c6d9f4b7a8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15e16a76dba440648ce59c6d9f4b7a8c" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 46267 } }
I20260811 06:29:32.965760 32603 raft_consensus.cc:388] T 00000000000000000000000000000000 P 15e16a76dba440648ce59c6d9f4b7a8c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:29:32.965919 32603 raft_consensus.cc:482] T 00000000000000000000000000000000 P 15e16a76dba440648ce59c6d9f4b7a8c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:29:32.966115 32603 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 15e16a76dba440648ce59c6d9f4b7a8c [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:29:32.968596 32603 raft_consensus.cc:504] T 00000000000000000000000000000000 P 15e16a76dba440648ce59c6d9f4b7a8c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15e16a76dba440648ce59c6d9f4b7a8c" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 46267 } }
I20260811 06:29:32.969041 32603 leader_election.cc:302] T 00000000000000000000000000000000 P 15e16a76dba440648ce59c6d9f4b7a8c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 15e16a76dba440648ce59c6d9f4b7a8c; no voters:
I20260811 06:29:32.969669 32603 leader_election.cc:288] T 00000000000000000000000000000000 P 15e16a76dba440648ce59c6d9f4b7a8c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:29:32.969841 32612 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 15e16a76dba440648ce59c6d9f4b7a8c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:29:32.971105 32612 raft_consensus.cc:686] T 00000000000000000000000000000000 P 15e16a76dba440648ce59c6d9f4b7a8c [term 1 LEADER]: Becoming Leader. State: Replica: 15e16a76dba440648ce59c6d9f4b7a8c, State: Running, Role: LEADER
I20260811 06:29:32.971760 32612 consensus_queue.cc:237] T 00000000000000000000000000000000 P 15e16a76dba440648ce59c6d9f4b7a8c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15e16a76dba440648ce59c6d9f4b7a8c" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 46267 } }
I20260811 06:29:32.971976 32603 sys_catalog.cc:564] T 00000000000000000000000000000000 P 15e16a76dba440648ce59c6d9f4b7a8c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:29:32.974829 32617 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15e16a76dba440648ce59c6d9f4b7a8c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 15e16a76dba440648ce59c6d9f4b7a8c. Latest consensus state: current_term: 1 leader_uuid: "15e16a76dba440648ce59c6d9f4b7a8c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15e16a76dba440648ce59c6d9f4b7a8c" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 46267 } } }
I20260811 06:29:32.975268 32617 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15e16a76dba440648ce59c6d9f4b7a8c [sys.catalog]: This master's current role is: LEADER
I20260811 06:29:32.975841 32616 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15e16a76dba440648ce59c6d9f4b7a8c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "15e16a76dba440648ce59c6d9f4b7a8c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15e16a76dba440648ce59c6d9f4b7a8c" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 46267 } } }
I20260811 06:29:32.976254 32616 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15e16a76dba440648ce59c6d9f4b7a8c [sys.catalog]: This master's current role is: LEADER
I20260811 06:29:32.980876 32626 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:29:32.987020 32626 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:29:32.995618 32626 catalog_manager.cc:1347] Generated new cluster ID: 636ec6ebe67a46efa91b976a8d0ab2d7
I20260811 06:29:32.995736 32626 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:29:33.004721 32626 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:29:33.006889 32626 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:29:33.019078 32626 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 15e16a76dba440648ce59c6d9f4b7a8c: Generated new TSK 0
I20260811 06:29:33.020357 32626 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.239.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.239.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.239.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.193.keytab.
Entry for principal kudu/127.28.239.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.193.keytab.
I20260811 06:29:33.085927 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.193:0
--local_ip_for_outbound_sockets=127.28.239.193
--webserver_interface=127.28.239.193
--webserver_port=0
--tserver_master_addrs=127.28.239.254:46267
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.193.keytab
--principal=kudu/127.28.239.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:42809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:29:33.437075 32645 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:29:33.437398 32645 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:29:33.437616 32645 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:29:33.445158 32645 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:29:33.445273 32645 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:29:33.445420 32645 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 06:29:33.445501 32645 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 06:29:33.445634 32645 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.193
I20260811 06:29:33.455910 32645 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:42809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.193.keytab
--principal=kudu/127.28.239.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.239.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.28.239.254:46267
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.193
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:29:33.457351 32645 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:29:33.458775 32645 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:29:33.472873 32659 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:29:33.472903 32655 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:29:33.472954 32656 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:29:33.473798 32645 server_base.cc:1034] running on GCE node
I20260811 06:29:33.474509 32645 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:29:33.475533 32645 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:29:33.476789 32645 hybrid_clock.cc:648] HybridClock initialized: now 1786429773476733 us; error 75 us; skew 500 ppm
I20260811 06:29:33.477135 32645 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:29:33 dist-test-slave-gkw7 krb5kdc[32451](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429773, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:29:33.492455 32645 init.cc:377] Logged in from keytab as kudu/127.28.239.193@KRBTEST.COM (short username kudu)
I20260811 06:29:33.494102 32645 webserver.cc:460] Webserver started at http://127.28.239.193:43531/ using document root <none> and password file <none>
I20260811 06:29:33.494813 32645 fs_manager.cc:359] Metadata directory not provided
I20260811 06:29:33.494918 32645 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:29:33.495221 32645 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:29:33.498519 32645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/minicluster-data/ts-0/data/instance:
uuid: "c7f16b4829bd44a9952c892f26d8e8c9"
format_stamp: "Formatted at 2026-08-11 06:29:33 on dist-test-slave-gkw7"
server_key: "b695c4689aad0e8fb93cdc6228078dfa"
server_key_iv: "197a702253445f6f4f0e6c8bc591a6f6"
server_key_version: "encryptionkey@0"
I20260811 06:29:33.499265 32645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/minicluster-data/ts-0/wal/instance:
uuid: "c7f16b4829bd44a9952c892f26d8e8c9"
format_stamp: "Formatted at 2026-08-11 06:29:33 on dist-test-slave-gkw7"
server_key: "b695c4689aad0e8fb93cdc6228078dfa"
server_key_iv: "197a702253445f6f4f0e6c8bc591a6f6"
server_key_version: "encryptionkey@0"
I20260811 06:29:33.504837 32645 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 06:29:33.508675 32672 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:29:33.510211 32645 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:29:33.510402 32645 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/minicluster-data/ts-0/wal
uuid: "c7f16b4829bd44a9952c892f26d8e8c9"
format_stamp: "Formatted at 2026-08-11 06:29:33 on dist-test-slave-gkw7"
server_key: "b695c4689aad0e8fb93cdc6228078dfa"
server_key_iv: "197a702253445f6f4f0e6c8bc591a6f6"
server_key_version: "encryptionkey@0"
I20260811 06:29:33.510624 32645 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:29:33.548615 32645 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:29:33.553283 32645 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:29:33.553620 32645 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:29:33.556439 32645 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:29:33.556579 32645 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:29:33.556717 32645 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:29:33.556813 32645 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 06:29:33 dist-test-slave-gkw7 krb5kdc[32451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429773, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.193@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:29:33.585027 32687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:33.557941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:46267 (local address 127.28.239.193:44789)
0811 06:29:33.558849 (+ 908us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:33.558908 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 06:29:33.561960 (+ 3052us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:29:33.563073 (+ 1113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:29:33.563089 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:29:33.563971 (+ 882us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:29:33.564923 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:33.564947 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:33.566257 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:33.566267 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:33.567608 (+ 1341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:33.567621 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:33.568892 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:33.568899 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:33.569107 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:33.570162 (+ 1055us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:29:33.570195 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:29:33.572903 (+ 2708us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:29:33.578047 (+ 5144us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:29:33.578060 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:29:33.578084 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:29:33.578861 (+ 777us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:29:33.579369 (+ 508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:29:33.579373 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:29:33.579376 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:29:33.579814 (+ 438us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:29:33.580805 (+ 991us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:29:33.580817 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:29:33.581477 (+ 660us) client_negotiation.cc:769] Sending connection context
0811 06:29:33.581994 (+ 517us) client_negotiation.cc:240] Negotiation successful
0811 06:29:33.582805 (+ 811us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":583,"thread_start_us":347,"threads_started":1}
I20260811 06:29:33.586616 32685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:33.557149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.193:44789 (local address 127.28.239.254:46267)
0811 06:29:33.557805 (+ 656us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:33.557813 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:33.559051 (+ 1238us) server_negotiation.cc:407] Connection header received
0811 06:29:33.562397 (+ 3346us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:33.562406 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:33.562517 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:33.562642 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:33.565113 (+ 2471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:33.566074 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:33.567769 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:33.568372 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:33.568520 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:33.573141 (+ 4621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:29:33.573147 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:29:33.573151 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:29:33.573187 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:29:33.577880 (+ 4693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:33.579013 (+ 1133us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:33.579019 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:33.579024 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:33.579220 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:33.580024 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:33.580033 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:33.580039 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:33.580501 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:29:33.580702 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:33.585788 (+ 5086us) server_negotiation.cc:299] Negotiation successful
0811 06:29:33.586103 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":515,"thread_start_us":396,"threads_started":1}
I20260811 06:29:33.612825 32645 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.193:39059
I20260811 06:29:33.612850 399 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.193:39059 every 8 connection(s)
I20260811 06:29:33.614465 32645 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
I20260811 06:29:33.617072 400 heartbeater.cc:347] Connected to a master server at 127.28.239.254:46267
I20260811 06:29:33.617797 400 heartbeater.cc:464] Registering TS with master...
I20260811 06:29:33.618722 400 heartbeater.cc:511] Master 127.28.239.254:46267 requested a full tablet report, sending...
I20260811 06:29:33.619804 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 32645
I20260811 06:29:33.620026 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/minicluster-data/ts-0/wal/instance
I20260811 06:29:33.620277 362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:33.602926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:46267 (local address 127.28.239.193:34905)
0811 06:29:33.603617 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:33.603635 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 06:29:33.603831 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:29:33.604340 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:29:33.604347 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:29:33.605250 (+ 903us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:29:33.605918 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:33.605940 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:33.607504 (+ 1564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:33.607516 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:33.609948 (+ 2432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:33.609973 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:33.610877 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:33.610886 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:33.611058 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:33.611993 (+ 935us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:29:33.612025 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:29:33.613409 (+ 1384us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:29:33.617293 (+ 3884us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:29:33.617298 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:29:33.617302 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:29:33.617976 (+ 674us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:29:33.618381 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:29:33.618385 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:29:33.618388 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:29:33.618549 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:29:33.619126 (+ 577us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:29:33.619131 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:29:33.619366 (+ 235us) client_negotiation.cc:769] Sending connection context
0811 06:29:33.619520 (+ 154us) client_negotiation.cc:240] Negotiation successful
0811 06:29:33.619815 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":371,"threads_started":1}
I20260811 06:29:33.620339 32685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:33.602925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.193:34905 (local address 127.28.239.254:46267)
0811 06:29:33.603129 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:33.603134 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:33.603749 (+ 615us) server_negotiation.cc:407] Connection header received
0811 06:29:33.604019 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:33.604024 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:33.604142 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:33.604275 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:33.606141 (+ 1866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:33.607316 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:33.610167 (+ 2851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:33.610660 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:33.610772 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:33.613653 (+ 2881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:29:33.613660 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:29:33.613663 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:29:33.613696 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:29:33.617158 (+ 3462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:33.618094 (+ 936us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:33.618099 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:33.618101 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:33.618242 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:33.618670 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:33.618674 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:33.618675 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:33.619001 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:29:33.619117 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:33.619682 (+ 565us) server_negotiation.cc:299] Negotiation successful
0811 06:29:33.619943 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 06:29:33.620806 29631 external_mini_cluster.cc:1428] Setting key 9cbfee42b08724a59316f648022da7d0
I20260811 06:29:33.623054 32527 ts_manager.cc:194] Registered new tserver with Master: c7f16b4829bd44a9952c892f26d8e8c9 (127.28.239.193:39059)
I20260811 06:29:33.625062 32527 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.239.193@KRBTEST.COM'} at 127.28.239.193:44789
WARNING: no policy specified for kudu/127.28.239.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.239.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.239.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.194.keytab.
Entry for principal kudu/127.28.239.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.194.keytab.
I20260811 06:29:33.669651 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.194:0
--local_ip_for_outbound_sockets=127.28.239.194
--webserver_interface=127.28.239.194
--webserver_port=0
--tserver_master_addrs=127.28.239.254:46267
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.194.keytab
--principal=kudu/127.28.239.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:42809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:29:34.013361 407 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:29:34.013684 407 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:29:34.013908 407 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:29:34.021217 407 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:29:34.021324 407 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:29:34.021517 407 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 06:29:34.021623 407 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 06:29:34.021762 407 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.194
I20260811 06:29:34.031721 407 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:42809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.194.keytab
--principal=kudu/127.28.239.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.239.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.28.239.254:46267
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.194
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:29:34.033102 407 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:29:34.034538 407 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:29:34.048368 414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:29:34.048396 413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:29:34.048441 417 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:29:34.049239 407 server_base.cc:1034] running on GCE node
I20260811 06:29:34.049980 407 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:29:34.051005 407 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:29:34.052253 407 hybrid_clock.cc:648] HybridClock initialized: now 1786429774052182 us; error 42 us; skew 500 ppm
I20260811 06:29:34.052632 407 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:29:34 dist-test-slave-gkw7 krb5kdc[32451](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429774, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:29:34.056136 407 init.cc:377] Logged in from keytab as kudu/127.28.239.194@KRBTEST.COM (short username kudu)
I20260811 06:29:34.057833 407 webserver.cc:460] Webserver started at http://127.28.239.194:45393/ using document root <none> and password file <none>
I20260811 06:29:34.058578 407 fs_manager.cc:359] Metadata directory not provided
I20260811 06:29:34.058717 407 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:29:34.059063 407 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:29:34.062285 407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/minicluster-data/ts-1/data/instance:
uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2"
format_stamp: "Formatted at 2026-08-11 06:29:34 on dist-test-slave-gkw7"
server_key: "72936739941072995470d52995ec3312"
server_key_iv: "66efbf36c95e741cc206c842f501beb4"
server_key_version: "encryptionkey@0"
I20260811 06:29:34.063108 407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/minicluster-data/ts-1/wal/instance:
uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2"
format_stamp: "Formatted at 2026-08-11 06:29:34 on dist-test-slave-gkw7"
server_key: "72936739941072995470d52995ec3312"
server_key_iv: "66efbf36c95e741cc206c842f501beb4"
server_key_version: "encryptionkey@0"
I20260811 06:29:34.068691 407 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.006s
I20260811 06:29:34.072366 430 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:29:34.073969 407 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 06:29:34.074172 407 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/minicluster-data/ts-1/wal
uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2"
format_stamp: "Formatted at 2026-08-11 06:29:34 on dist-test-slave-gkw7"
server_key: "72936739941072995470d52995ec3312"
server_key_iv: "66efbf36c95e741cc206c842f501beb4"
server_key_version: "encryptionkey@0"
I20260811 06:29:34.074389 407 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:29:34.096078 407 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:29:34.100566 407 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:29:34.100930 407 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:29:34.103786 407 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:29:34.103945 407 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:29:34.104090 407 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:29:34.104197 407 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 06:29:34 dist-test-slave-gkw7 krb5kdc[32451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429774, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.194@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:29:34.128137 32685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:34.104585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.194:59935 (local address 127.28.239.254:46267)
0811 06:29:34.104810 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:34.104818 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:34.106454 (+ 1636us) server_negotiation.cc:407] Connection header received
0811 06:29:34.108727 (+ 2273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:34.108737 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:34.108854 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:34.108991 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:34.111351 (+ 2360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:34.112284 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:34.115097 (+ 2813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:34.115843 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:34.116009 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:34.120523 (+ 4514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:29:34.120532 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:29:34.120537 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:29:34.120597 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:29:34.124481 (+ 3884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:34.125465 (+ 984us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:34.125471 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:34.125475 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:34.125701 (+ 226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:34.126279 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:34.126284 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:34.126288 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:34.126601 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:29:34.126777 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:34.127539 (+ 762us) server_negotiation.cc:299] Negotiation successful
0811 06:29:34.127772 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 06:29:34.129211 446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:34.105268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:46267 (local address 127.28.239.194:59935)
0811 06:29:34.106228 (+ 960us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:34.106290 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 06:29:34.108320 (+ 2030us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:29:34.109314 (+ 994us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:29:34.109323 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:29:34.110208 (+ 885us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:29:34.111139 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:34.111173 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:34.112448 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:34.112458 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:34.114854 (+ 2396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:34.114868 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:34.116148 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:34.116157 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:34.116430 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:34.117219 (+ 789us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:29:34.117246 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:29:34.120197 (+ 2951us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:29:34.124635 (+ 4438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:29:34.124645 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:29:34.124667 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:29:34.125323 (+ 656us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:29:34.125871 (+ 548us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:29:34.125878 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:29:34.125884 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:29:34.126131 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:29:34.126739 (+ 608us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:29:34.126747 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:29:34.127145 (+ 398us) client_negotiation.cc:769] Sending connection context
0811 06:29:34.127430 (+ 285us) client_negotiation.cc:240] Negotiation successful
0811 06:29:34.128000 (+ 570us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":682,"thread_start_us":429,"threads_started":1}
I20260811 06:29:34.150520 32685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:34.135350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.194:46649 (local address 127.28.239.254:46267)
0811 06:29:34.135519 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:34.135523 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:34.136182 (+ 659us) server_negotiation.cc:407] Connection header received
0811 06:29:34.136390 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:34.136395 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:34.136465 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:34.136570 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:34.137524 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:34.138522 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:34.140805 (+ 2283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:34.141284 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:34.141425 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:34.143475 (+ 2050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:29:34.143481 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:29:34.143484 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:29:34.143515 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:29:34.146825 (+ 3310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:34.147965 (+ 1140us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:34.147974 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:34.147980 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:34.148157 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:34.148691 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:34.148699 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:34.148705 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:34.149299 (+ 594us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:29:34.149517 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:34.149955 (+ 438us) server_negotiation.cc:299] Negotiation successful
0811 06:29:34.150176 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 06:29:34.150656 536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:34.135434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:46267 (local address 127.28.239.194:46649)
0811 06:29:34.136070 (+ 636us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:34.136086 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:29:34.136254 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:29:34.136600 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:29:34.136605 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:29:34.137078 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:29:34.137375 (+ 297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:34.137385 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:34.138755 (+ 1370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:34.138774 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:34.140573 (+ 1799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:34.140597 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:34.141509 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:34.141516 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:34.141761 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:34.142496 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:29:34.142513 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:29:34.143252 (+ 739us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:29:34.146985 (+ 3733us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:29:34.146995 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:29:34.147001 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:29:34.147813 (+ 812us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:29:34.148293 (+ 480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:29:34.148299 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:29:34.148302 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:29:34.148528 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:29:34.149442 (+ 914us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:29:34.149446 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:29:34.149668 (+ 222us) client_negotiation.cc:769] Sending connection context
0811 06:29:34.149854 (+ 186us) client_negotiation.cc:240] Negotiation successful
0811 06:29:34.150203 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":375,"threads_started":1}
I20260811 06:29:34.162461 407 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.194:33025
I20260811 06:29:34.162549 617 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.194:33025 every 8 connection(s)
I20260811 06:29:34.163832 407 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
I20260811 06:29:34.166172 618 heartbeater.cc:347] Connected to a master server at 127.28.239.254:46267
I20260811 06:29:34.166441 618 heartbeater.cc:464] Registering TS with master...
I20260811 06:29:34.167147 618 heartbeater.cc:511] Master 127.28.239.254:46267 requested a full tablet report, sending...
I20260811 06:29:34.168550 32527 ts_manager.cc:194] Registered new tserver with Master: 47f4dcc280824ca09b9c04fe9ee5c3a2 (127.28.239.194:33025)
I20260811 06:29:34.169343 32527 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.239.194@KRBTEST.COM'} at 127.28.239.194:59935
I20260811 06:29:34.172501 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 407
I20260811 06:29:34.172654 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/minicluster-data/ts-1/wal/instance
I20260811 06:29:34.173130 29631 external_mini_cluster.cc:1428] Setting key 58b94d13be3a58b37e5aff03bfc61938
WARNING: no policy specified for kudu/127.28.239.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.239.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.239.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.195.keytab.
Entry for principal kudu/127.28.239.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.195.keytab.
I20260811 06:29:34.220813 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.195:0
--local_ip_for_outbound_sockets=127.28.239.195
--webserver_interface=127.28.239.195
--webserver_port=0
--tserver_master_addrs=127.28.239.254:46267
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.195.keytab
--principal=kudu/127.28.239.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:42809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:29:34.566787 622 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:29:34.567139 622 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:29:34.567322 622 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:29:34.575104 622 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:29:34.575263 622 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:29:34.575405 622 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 06:29:34.575495 622 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 06:29:34.575628 622 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.195
I20260811 06:29:34.586338 622 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:42809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.195.keytab
--principal=kudu/127.28.239.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.239.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.28.239.254:46267
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.195
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:29:34.587710 622 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:29:34.589170 622 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:29:34.603260 633 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:29:34.603312 631 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:29:34.603338 629 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:29:34.604276 622 server_base.cc:1034] running on GCE node
I20260811 06:29:34.604920 622 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:29:34.605950 622 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:29:34.607183 622 hybrid_clock.cc:648] HybridClock initialized: now 1786429774607122 us; error 55 us; skew 500 ppm
I20260811 06:29:34.607546 622 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:29:34 dist-test-slave-gkw7 krb5kdc[32451](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429774, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:29:34.611019 622 init.cc:377] Logged in from keytab as kudu/127.28.239.195@KRBTEST.COM (short username kudu)
I20260811 06:29:34.612805 622 webserver.cc:460] Webserver started at http://127.28.239.195:39523/ using document root <none> and password file <none>
I20260811 06:29:34.613878 622 fs_manager.cc:359] Metadata directory not provided
I20260811 06:29:34.614027 622 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:29:34.614377 622 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:29:34.618636 622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/minicluster-data/ts-2/data/instance:
uuid: "d184d745e54d4f629f9b89868405f29d"
format_stamp: "Formatted at 2026-08-11 06:29:34 on dist-test-slave-gkw7"
server_key: "dd36d789f5b1ec42f8217e11df5fe678"
server_key_iv: "a4428aeabc372d3002ad3ff7255142b2"
server_key_version: "encryptionkey@0"
I20260811 06:29:34.619544 622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/minicluster-data/ts-2/wal/instance:
uuid: "d184d745e54d4f629f9b89868405f29d"
format_stamp: "Formatted at 2026-08-11 06:29:34 on dist-test-slave-gkw7"
server_key: "dd36d789f5b1ec42f8217e11df5fe678"
server_key_iv: "a4428aeabc372d3002ad3ff7255142b2"
server_key_version: "encryptionkey@0"
I20260811 06:29:34.625041 622 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260811 06:29:34.628847 400 heartbeater.cc:503] Master 127.28.239.254:46267 was elected leader, sending a full tablet report...
I20260811 06:29:34.629199 643 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:29:34.631311 622 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.003s
I20260811 06:29:34.631521 622 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/minicluster-data/ts-2/wal
uuid: "d184d745e54d4f629f9b89868405f29d"
format_stamp: "Formatted at 2026-08-11 06:29:34 on dist-test-slave-gkw7"
server_key: "dd36d789f5b1ec42f8217e11df5fe678"
server_key_iv: "a4428aeabc372d3002ad3ff7255142b2"
server_key_version: "encryptionkey@0"
I20260811 06:29:34.631755 622 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:29:34.700721 622 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:29:34.705422 622 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:29:34.705754 622 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:29:34.708612 622 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:29:34.708760 622 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:29:34.708884 622 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:29:34.708971 622 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 06:29:34 dist-test-slave-gkw7 krb5kdc[32451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429774, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.195@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:29:34.738536 655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:34.709670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.195:58765 (local address 127.28.239.254:46267)
0811 06:29:34.712143 (+ 2473us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:34.712153 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:34.712182 (+ 29us) server_negotiation.cc:407] Connection header received
0811 06:29:34.715015 (+ 2833us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:34.715025 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:34.715147 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:34.715320 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:34.718653 (+ 3333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:34.719577 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:34.721933 (+ 2356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:34.722711 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:34.722896 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:34.729302 (+ 6406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:29:34.729314 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:29:34.729321 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:29:34.729384 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:29:34.734073 (+ 4689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:34.735424 (+ 1351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:34.735433 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:34.735439 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:34.735630 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:34.736253 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:34.736261 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:34.736267 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:34.736797 (+ 530us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:29:34.736994 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:34.737898 (+ 904us) server_negotiation.cc:299] Negotiation successful
0811 06:29:34.738137 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2334,"thread_start_us":2240,"threads_started":1}
I20260811 06:29:34.739701 656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:34.710267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:46267 (local address 127.28.239.195:58765)
0811 06:29:34.711568 (+ 1301us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:34.711656 (+ 88us) client_negotiation.cc:174] Beginning negotiation
0811 06:29:34.714583 (+ 2927us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:29:34.715809 (+ 1226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:29:34.715823 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:29:34.716921 (+ 1098us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:29:34.718371 (+ 1450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:34.718414 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:34.719761 (+ 1347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:34.719768 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:34.721697 (+ 1929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:34.721717 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:34.723034 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:34.723045 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:34.723415 (+ 370us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:34.724383 (+ 968us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:29:34.724425 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:29:34.728931 (+ 4506us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:29:34.734260 (+ 5329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:29:34.734278 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:29:34.734305 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:29:34.735214 (+ 909us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:29:34.735775 (+ 561us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:29:34.735779 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:29:34.735782 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:29:34.736109 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:29:34.736966 (+ 857us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:29:34.736972 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:29:34.737387 (+ 415us) client_negotiation.cc:769] Sending connection context
0811 06:29:34.737777 (+ 390us) client_negotiation.cc:240] Negotiation successful
0811 06:29:34.738394 (+ 617us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":816,"thread_start_us":484,"threads_started":1}
I20260811 06:29:34.763767 763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:34.746588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:46267 (local address 127.28.239.195:41195)
0811 06:29:34.747180 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:34.747200 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 06:29:34.747380 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:29:34.747738 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:29:34.747742 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:29:34.748474 (+ 732us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:29:34.749064 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:34.749085 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:34.750484 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:34.750488 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:34.752306 (+ 1818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:34.752323 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:34.753752 (+ 1429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:34.753763 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:34.753967 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:34.754681 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:29:34.754713 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:29:34.755919 (+ 1206us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:29:34.760560 (+ 4641us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:29:34.760568 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:29:34.760574 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:29:34.761108 (+ 534us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:29:34.761688 (+ 580us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:29:34.761702 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:29:34.761705 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:29:34.761921 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:29:34.762634 (+ 713us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:29:34.762642 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:29:34.762884 (+ 242us) client_negotiation.cc:769] Sending connection context
0811 06:29:34.763002 (+ 118us) client_negotiation.cc:240] Negotiation successful
0811 06:29:34.763249 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":338,"threads_started":1}
I20260811 06:29:34.763756 655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:34.746297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.195:41195 (local address 127.28.239.254:46267)
0811 06:29:34.746518 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:34.746523 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:34.747301 (+ 778us) server_negotiation.cc:407] Connection header received
0811 06:29:34.747529 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:34.747535 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:34.747604 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:34.747707 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:34.749276 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:34.750285 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:34.752514 (+ 2229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:34.753447 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:34.753669 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:34.756109 (+ 2440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:29:34.756118 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:29:34.756122 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:29:34.756163 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:29:34.760406 (+ 4243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:34.761315 (+ 909us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:34.761323 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:34.761328 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:34.761498 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:34.762061 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:34.762068 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:34.762074 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:34.762483 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:29:34.762661 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:34.763149 (+ 488us) server_negotiation.cc:299] Negotiation successful
0811 06:29:34.763384 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 06:29:34.765763 622 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.195:42143
I20260811 06:29:34.765893 826 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.195:42143 every 8 connection(s)
I20260811 06:29:34.767094 622 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
I20260811 06:29:34.769304 830 heartbeater.cc:347] Connected to a master server at 127.28.239.254:46267
I20260811 06:29:34.769590 830 heartbeater.cc:464] Registering TS with master...
I20260811 06:29:34.770203 830 heartbeater.cc:511] Master 127.28.239.254:46267 requested a full tablet report, sending...
I20260811 06:29:34.771430 32527 ts_manager.cc:194] Registered new tserver with Master: d184d745e54d4f629f9b89868405f29d (127.28.239.195:42143)
I20260811 06:29:34.772223 32527 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.239.195@KRBTEST.COM'} at 127.28.239.195:58765
I20260811 06:29:34.773931 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 622
I20260811 06:29:34.774133 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/minicluster-data/ts-2/wal/instance
I20260811 06:29:34.774576 29631 external_mini_cluster.cc:1428] Setting key f71cfda3df9bc668d20b543bf575cc52
I20260811 06:29:34.777364 29631 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 06:29:34 dist-test-slave-gkw7 krb5kdc[32451](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429774, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 06:29:34 dist-test-slave-gkw7 krb5kdc[32451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429774, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:29:34.830730 655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:34.817156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50278 (local address 127.28.239.254:46267)
0811 06:29:34.817334 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:34.817339 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:34.817604 (+ 265us) server_negotiation.cc:407] Connection header received
0811 06:29:34.817825 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:34.817829 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:34.817895 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:34.817988 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:34.819351 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:34.820087 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:34.821260 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:34.821725 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:34.821841 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:34.825631 (+ 3790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:29:34.825636 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:29:34.825639 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:29:34.825672 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:29:34.828196 (+ 2524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:34.828917 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:34.828920 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:34.828923 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:34.829052 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:34.829442 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:34.829445 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:34.829447 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:34.829775 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:29:34.829970 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:34.830204 (+ 234us) server_negotiation.cc:299] Negotiation successful
0811 06:29:34.830365 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 06:29:34.834868 32527 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50278:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:29:34.842120 32527 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:29:34.880224 849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:34.855654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44830 (local address 127.28.239.194:33025)
0811 06:29:34.868746 (+ 13092us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:34.868754 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:34.869046 (+ 292us) server_negotiation.cc:407] Connection header received
0811 06:29:34.869874 (+ 828us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:34.869886 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:34.870233 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:34.870452 (+ 219us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:29:34.871547 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:34.873665 (+ 2118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:34.877273 (+ 3608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:34.878795 (+ 1522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:34.879004 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:34.879198 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:34.879728 (+ 530us) server_negotiation.cc:299] Negotiation successful
0811 06:29:34.879918 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12913,"thread_start_us":12780,"threads_started":1}
I20260811 06:29:34.880323 850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:34.855761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35512 (local address 127.28.239.195:42143)
0811 06:29:34.868453 (+ 12692us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:34.868465 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:34.868852 (+ 387us) server_negotiation.cc:407] Connection header received
0811 06:29:34.869252 (+ 400us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:34.869260 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:34.869562 (+ 302us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:34.869795 (+ 233us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:29:34.871188 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:34.873622 (+ 2434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:34.876792 (+ 3170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:34.878455 (+ 1663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:34.878660 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:34.878877 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:34.879830 (+ 953us) server_negotiation.cc:299] Negotiation successful
0811 06:29:34.879983 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12491,"thread_start_us":12304,"threads_started":1}
I20260811 06:29:34.880369 846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:34.855375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.194:33025 (local address 127.0.0.1:44830)
0811 06:29:34.868907 (+ 13532us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:34.868932 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 06:29:34.869077 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:29:34.870736 (+ 1659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:29:34.870748 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:29:34.870785 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:29:34.871310 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:34.871326 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:34.873871 (+ 2545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:34.873880 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:34.877098 (+ 3218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:34.877113 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:34.879109 (+ 1996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:34.879116 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:34.879511 (+ 395us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:34.879537 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 06:29:34.879670 (+ 133us) client_negotiation.cc:240] Negotiation successful
0811 06:29:34.879900 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13299,"mutex_wait_us":69,"thread_start_us":12265,"threads_started":1}
I20260811 06:29:34.880998 847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:34.855224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.195:42143 (local address 127.0.0.1:35512)
0811 06:29:34.868664 (+ 13440us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:34.868710 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 06:29:34.868978 (+ 268us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:29:34.870227 (+ 1249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:29:34.870239 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:29:34.870289 (+ 50us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:29:34.870886 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:34.870904 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:34.873838 (+ 2934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:34.873847 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:34.876604 (+ 2757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:34.876626 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:34.879034 (+ 2408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:34.879042 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:34.879454 (+ 412us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:34.879517 (+ 63us) client_negotiation.cc:769] Sending connection context
0811 06:29:34.879670 (+ 153us) client_negotiation.cc:240] Negotiation successful
0811 06:29:34.880653 (+ 983us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12584,"mutex_wait_us":895,"thread_start_us":12542,"threads_started":1}
I20260811 06:29:34.884312 507 tablet_service.cc:1467] Processing CreateTablet for tablet 6f9ef3df97c74376bce188bdf6bf0c09 (DEFAULT_TABLE table=test-table [id=70e2a79884c2478899214d69e382a7d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:29:34.884649 723 tablet_service.cc:1467] Processing CreateTablet for tablet 6f9ef3df97c74376bce188bdf6bf0c09 (DEFAULT_TABLE table=test-table [id=70e2a79884c2478899214d69e382a7d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:29:34.885809 851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:34.869383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.193:39059 (local address 127.0.0.1:34340)
0811 06:29:34.874497 (+ 5114us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:34.874515 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 06:29:34.874646 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:29:34.876396 (+ 1750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:29:34.876403 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:29:34.876434 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:29:34.876966 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:34.876975 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:34.879253 (+ 2278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:34.879258 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:34.882910 (+ 3652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:34.882926 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:34.884924 (+ 1998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:34.884935 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:34.885108 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:34.885148 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 06:29:34.885255 (+ 107us) client_negotiation.cc:240] Negotiation successful
0811 06:29:34.885389 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4990,"mutex_wait_us":33,"thread_start_us":4831,"threads_started":1}
I20260811 06:29:34.886251 723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f9ef3df97c74376bce188bdf6bf0c09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:29:34.886298 507 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f9ef3df97c74376bce188bdf6bf0c09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:29:34.886494 853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:34.869495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34340 (local address 127.28.239.193:39059)
0811 06:29:34.875783 (+ 6288us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:34.875794 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:34.875826 (+ 32us) server_negotiation.cc:407] Connection header received
0811 06:29:34.875927 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:34.875934 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:34.876178 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:34.876374 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:29:34.877149 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:34.879008 (+ 1859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:34.883090 (+ 4082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:34.884584 (+ 1494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:34.884811 (+ 227us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:34.884980 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:34.885324 (+ 344us) server_negotiation.cc:299] Negotiation successful
0811 06:29:34.885510 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6123,"thread_start_us":5955,"threads_started":1}
I20260811 06:29:34.890375 32756 tablet_service.cc:1467] Processing CreateTablet for tablet 6f9ef3df97c74376bce188bdf6bf0c09 (DEFAULT_TABLE table=test-table [id=70e2a79884c2478899214d69e382a7d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:29:34.891830 32756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f9ef3df97c74376bce188bdf6bf0c09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:29:34.898747 854 tablet_bootstrap.cc:492] T 6f9ef3df97c74376bce188bdf6bf0c09 P 47f4dcc280824ca09b9c04fe9ee5c3a2: Bootstrap starting.
I20260811 06:29:34.902041 854 tablet_bootstrap.cc:654] T 6f9ef3df97c74376bce188bdf6bf0c09 P 47f4dcc280824ca09b9c04fe9ee5c3a2: Neither blocks nor log segments found. Creating new log.
I20260811 06:29:34.903115 854 log.cc:824] T 6f9ef3df97c74376bce188bdf6bf0c09 P 47f4dcc280824ca09b9c04fe9ee5c3a2: Log is configured to *not* fsync() on all Append() calls
I20260811 06:29:34.905422 855 tablet_bootstrap.cc:492] T 6f9ef3df97c74376bce188bdf6bf0c09 P d184d745e54d4f629f9b89868405f29d: Bootstrap starting.
I20260811 06:29:34.906543 854 tablet_bootstrap.cc:492] T 6f9ef3df97c74376bce188bdf6bf0c09 P 47f4dcc280824ca09b9c04fe9ee5c3a2: No bootstrap required, opened a new log
I20260811 06:29:34.906926 854 ts_tablet_manager.cc:1397] T 6f9ef3df97c74376bce188bdf6bf0c09 P 47f4dcc280824ca09b9c04fe9ee5c3a2: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260811 06:29:34.907124 862 tablet_bootstrap.cc:492] T 6f9ef3df97c74376bce188bdf6bf0c09 P c7f16b4829bd44a9952c892f26d8e8c9: Bootstrap starting.
I20260811 06:29:34.909286 855 tablet_bootstrap.cc:654] T 6f9ef3df97c74376bce188bdf6bf0c09 P d184d745e54d4f629f9b89868405f29d: Neither blocks nor log segments found. Creating new log.
I20260811 06:29:34.910208 862 tablet_bootstrap.cc:654] T 6f9ef3df97c74376bce188bdf6bf0c09 P c7f16b4829bd44a9952c892f26d8e8c9: Neither blocks nor log segments found. Creating new log.
I20260811 06:29:34.910593 855 log.cc:824] T 6f9ef3df97c74376bce188bdf6bf0c09 P d184d745e54d4f629f9b89868405f29d: Log is configured to *not* fsync() on all Append() calls
I20260811 06:29:34.911346 862 log.cc:824] T 6f9ef3df97c74376bce188bdf6bf0c09 P c7f16b4829bd44a9952c892f26d8e8c9: Log is configured to *not* fsync() on all Append() calls
I20260811 06:29:34.913873 855 tablet_bootstrap.cc:492] T 6f9ef3df97c74376bce188bdf6bf0c09 P d184d745e54d4f629f9b89868405f29d: No bootstrap required, opened a new log
I20260811 06:29:34.914146 855 ts_tablet_manager.cc:1397] T 6f9ef3df97c74376bce188bdf6bf0c09 P d184d745e54d4f629f9b89868405f29d: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260811 06:29:34.916440 862 tablet_bootstrap.cc:492] T 6f9ef3df97c74376bce188bdf6bf0c09 P c7f16b4829bd44a9952c892f26d8e8c9: No bootstrap required, opened a new log
I20260811 06:29:34.916764 862 ts_tablet_manager.cc:1397] T 6f9ef3df97c74376bce188bdf6bf0c09 P c7f16b4829bd44a9952c892f26d8e8c9: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.000s
I20260811 06:29:34.917798 854 raft_consensus.cc:348] T 6f9ef3df97c74376bce188bdf6bf0c09 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d184d745e54d4f629f9b89868405f29d" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 42143 } } peers { permanent_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 33025 } } peers { permanent_uuid: "c7f16b4829bd44a9952c892f26d8e8c9" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 39059 } }
I20260811 06:29:34.918258 854 raft_consensus.cc:374] T 6f9ef3df97c74376bce188bdf6bf0c09 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:29:34.918380 854 raft_consensus.cc:729] T 6f9ef3df97c74376bce188bdf6bf0c09 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47f4dcc280824ca09b9c04fe9ee5c3a2, State: Initialized, Role: FOLLOWER
I20260811 06:29:34.919024 854 consensus_queue.cc:260] T 6f9ef3df97c74376bce188bdf6bf0c09 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d184d745e54d4f629f9b89868405f29d" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 42143 } } peers { permanent_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 33025 } } peers { permanent_uuid: "c7f16b4829bd44a9952c892f26d8e8c9" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 39059 } }
I20260811 06:29:34.920298 618 heartbeater.cc:503] Master 127.28.239.254:46267 was elected leader, sending a full tablet report...
I20260811 06:29:34.920879 854 ts_tablet_manager.cc:1428] T 6f9ef3df97c74376bce188bdf6bf0c09 P 47f4dcc280824ca09b9c04fe9ee5c3a2: Time spent starting tablet: real 0.014s user 0.007s sys 0.005s
I20260811 06:29:34.924063 855 raft_consensus.cc:348] T 6f9ef3df97c74376bce188bdf6bf0c09 P d184d745e54d4f629f9b89868405f29d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d184d745e54d4f629f9b89868405f29d" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 42143 } } peers { permanent_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 33025 } } peers { permanent_uuid: "c7f16b4829bd44a9952c892f26d8e8c9" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 39059 } }
I20260811 06:29:34.924446 855 raft_consensus.cc:374] T 6f9ef3df97c74376bce188bdf6bf0c09 P d184d745e54d4f629f9b89868405f29d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:29:34.924575 855 raft_consensus.cc:729] T 6f9ef3df97c74376bce188bdf6bf0c09 P d184d745e54d4f629f9b89868405f29d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d184d745e54d4f629f9b89868405f29d, State: Initialized, Role: FOLLOWER
I20260811 06:29:34.925326 855 consensus_queue.cc:260] T 6f9ef3df97c74376bce188bdf6bf0c09 P d184d745e54d4f629f9b89868405f29d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d184d745e54d4f629f9b89868405f29d" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 42143 } } peers { permanent_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 33025 } } peers { permanent_uuid: "c7f16b4829bd44a9952c892f26d8e8c9" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 39059 } }
I20260811 06:29:34.926753 830 heartbeater.cc:503] Master 127.28.239.254:46267 was elected leader, sending a full tablet report...
I20260811 06:29:34.928732 862 raft_consensus.cc:348] T 6f9ef3df97c74376bce188bdf6bf0c09 P c7f16b4829bd44a9952c892f26d8e8c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d184d745e54d4f629f9b89868405f29d" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 42143 } } peers { permanent_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 33025 } } peers { permanent_uuid: "c7f16b4829bd44a9952c892f26d8e8c9" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 39059 } }
I20260811 06:29:34.929121 855 ts_tablet_manager.cc:1428] T 6f9ef3df97c74376bce188bdf6bf0c09 P d184d745e54d4f629f9b89868405f29d: Time spent starting tablet: real 0.015s user 0.012s sys 0.000s
I20260811 06:29:34.929183 862 raft_consensus.cc:374] T 6f9ef3df97c74376bce188bdf6bf0c09 P c7f16b4829bd44a9952c892f26d8e8c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:29:34.929343 862 raft_consensus.cc:729] T 6f9ef3df97c74376bce188bdf6bf0c09 P c7f16b4829bd44a9952c892f26d8e8c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7f16b4829bd44a9952c892f26d8e8c9, State: Initialized, Role: FOLLOWER
I20260811 06:29:34.930176 862 consensus_queue.cc:260] T 6f9ef3df97c74376bce188bdf6bf0c09 P c7f16b4829bd44a9952c892f26d8e8c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d184d745e54d4f629f9b89868405f29d" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 42143 } } peers { permanent_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 33025 } } peers { permanent_uuid: "c7f16b4829bd44a9952c892f26d8e8c9" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 39059 } }
I20260811 06:29:34.932341 862 ts_tablet_manager.cc:1428] T 6f9ef3df97c74376bce188bdf6bf0c09 P c7f16b4829bd44a9952c892f26d8e8c9: Time spent starting tablet: real 0.015s user 0.016s sys 0.000s
W20260811 06:29:35.018940 832 tablet.cc:2369] T 6f9ef3df97c74376bce188bdf6bf0c09 P d184d745e54d4f629f9b89868405f29d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:29:35.019882 866 raft_consensus.cc:482] T 6f9ef3df97c74376bce188bdf6bf0c09 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:29:35.020227 866 raft_consensus.cc:504] T 6f9ef3df97c74376bce188bdf6bf0c09 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d184d745e54d4f629f9b89868405f29d" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 42143 } } peers { permanent_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 33025 } } peers { permanent_uuid: "c7f16b4829bd44a9952c892f26d8e8c9" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 39059 } }
I20260811 06:29:35.022074 866 leader_election.cc:288] T 6f9ef3df97c74376bce188bdf6bf0c09 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d184d745e54d4f629f9b89868405f29d (127.28.239.195:42143), c7f16b4829bd44a9952c892f26d8e8c9 (127.28.239.193:39059)
I20260811 06:29:35.029466 850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:35.022671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.194:40855 (local address 127.28.239.195:42143)
0811 06:29:35.022893 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:35.022898 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:35.023412 (+ 514us) server_negotiation.cc:407] Connection header received
0811 06:29:35.023625 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:35.023632 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:35.023717 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:35.023837 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:29:35.024491 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:35.025211 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:35.027231 (+ 2020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:35.028364 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:35.028562 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:35.028657 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:35.029013 (+ 356us) server_negotiation.cc:299] Negotiation successful
0811 06:29:35.029178 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260811 06:29:35.029600 869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:35.022483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.195:42143 (local address 127.28.239.194:40855)
0811 06:29:35.023268 (+ 785us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:35.023297 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0811 06:29:35.023452 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:29:35.023943 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:29:35.023948 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:29:35.023979 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:29:35.024332 (+ 353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:35.024342 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:35.025365 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:35.025373 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:35.027000 (+ 1627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:35.027015 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:35.028618 (+ 1603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:35.028627 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:35.028815 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:35.028851 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 06:29:35.028956 (+ 105us) client_negotiation.cc:240] Negotiation successful
0811 06:29:35.029096 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":603,"mutex_wait_us":1,"thread_start_us":402,"threads_started":1}
I20260811 06:29:35.030643 853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:35.023775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.194:39413 (local address 127.28.239.193:39059)
0811 06:29:35.024093 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:35.024101 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:35.024201 (+ 100us) server_negotiation.cc:407] Connection header received
0811 06:29:35.024433 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:35.024438 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:35.024498 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:35.024603 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:29:35.025168 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:35.025994 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:35.027762 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:35.028859 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:35.029018 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:35.029085 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:35.030264 (+ 1179us) server_negotiation.cc:299] Negotiation successful
0811 06:29:35.030400 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156}
I20260811 06:29:35.030694 870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:35.023407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.193:39059 (local address 127.28.239.194:39413)
0811 06:29:35.024069 (+ 662us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:35.024091 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 06:29:35.024276 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:29:35.024624 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:29:35.024629 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:29:35.024652 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:29:35.025000 (+ 348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:35.025008 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:35.026156 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:35.026165 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:35.027593 (+ 1428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:35.027605 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:35.029266 (+ 1661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:35.029274 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:35.029454 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:35.029498 (+ 44us) client_negotiation.cc:769] Sending connection context
0811 06:29:35.030229 (+ 731us) client_negotiation.cc:240] Negotiation successful
0811 06:29:35.030366 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":537,"thread_start_us":346,"threads_started":1}
I20260811 06:29:35.030704 762 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f9ef3df97c74376bce188bdf6bf0c09" candidate_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d184d745e54d4f629f9b89868405f29d" is_pre_election: true
I20260811 06:29:35.031472 762 raft_consensus.cc:2454] T 6f9ef3df97c74376bce188bdf6bf0c09 P d184d745e54d4f629f9b89868405f29d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 47f4dcc280824ca09b9c04fe9ee5c3a2 in term 0.
I20260811 06:29:35.031797 321 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f9ef3df97c74376bce188bdf6bf0c09" candidate_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7f16b4829bd44a9952c892f26d8e8c9" is_pre_election: true
I20260811 06:29:35.032425 321 raft_consensus.cc:2454] T 6f9ef3df97c74376bce188bdf6bf0c09 P c7f16b4829bd44a9952c892f26d8e8c9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 47f4dcc280824ca09b9c04fe9ee5c3a2 in term 0.
I20260811 06:29:35.032284 431 leader_election.cc:302] T 6f9ef3df97c74376bce188bdf6bf0c09 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47f4dcc280824ca09b9c04fe9ee5c3a2, d184d745e54d4f629f9b89868405f29d; no voters:
I20260811 06:29:35.032975 866 raft_consensus.cc:2781] T 6f9ef3df97c74376bce188bdf6bf0c09 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:29:35.033143 866 raft_consensus.cc:482] T 6f9ef3df97c74376bce188bdf6bf0c09 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:29:35.033300 866 raft_consensus.cc:3037] T 6f9ef3df97c74376bce188bdf6bf0c09 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:29:35.035765 866 raft_consensus.cc:504] T 6f9ef3df97c74376bce188bdf6bf0c09 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d184d745e54d4f629f9b89868405f29d" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 42143 } } peers { permanent_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 33025 } } peers { permanent_uuid: "c7f16b4829bd44a9952c892f26d8e8c9" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 39059 } }
I20260811 06:29:35.036406 866 leader_election.cc:288] T 6f9ef3df97c74376bce188bdf6bf0c09 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [CANDIDATE]: Term 1 election: Requested vote from peers d184d745e54d4f629f9b89868405f29d (127.28.239.195:42143), c7f16b4829bd44a9952c892f26d8e8c9 (127.28.239.193:39059)
I20260811 06:29:35.036912 762 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f9ef3df97c74376bce188bdf6bf0c09" candidate_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d184d745e54d4f629f9b89868405f29d"
I20260811 06:29:35.036952 321 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f9ef3df97c74376bce188bdf6bf0c09" candidate_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7f16b4829bd44a9952c892f26d8e8c9"
I20260811 06:29:35.037135 762 raft_consensus.cc:3037] T 6f9ef3df97c74376bce188bdf6bf0c09 P d184d745e54d4f629f9b89868405f29d [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:29:35.037159 321 raft_consensus.cc:3037] T 6f9ef3df97c74376bce188bdf6bf0c09 P c7f16b4829bd44a9952c892f26d8e8c9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:29:35.039511 762 raft_consensus.cc:2454] T 6f9ef3df97c74376bce188bdf6bf0c09 P d184d745e54d4f629f9b89868405f29d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47f4dcc280824ca09b9c04fe9ee5c3a2 in term 1.
I20260811 06:29:35.039527 321 raft_consensus.cc:2454] T 6f9ef3df97c74376bce188bdf6bf0c09 P c7f16b4829bd44a9952c892f26d8e8c9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47f4dcc280824ca09b9c04fe9ee5c3a2 in term 1.
I20260811 06:29:35.039964 431 leader_election.cc:302] T 6f9ef3df97c74376bce188bdf6bf0c09 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47f4dcc280824ca09b9c04fe9ee5c3a2, c7f16b4829bd44a9952c892f26d8e8c9; no voters:
I20260811 06:29:35.040503 866 raft_consensus.cc:2781] T 6f9ef3df97c74376bce188bdf6bf0c09 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:29:35.041043 866 raft_consensus.cc:686] T 6f9ef3df97c74376bce188bdf6bf0c09 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [term 1 LEADER]: Becoming Leader. State: Replica: 47f4dcc280824ca09b9c04fe9ee5c3a2, State: Running, Role: LEADER
I20260811 06:29:35.041683 866 consensus_queue.cc:237] T 6f9ef3df97c74376bce188bdf6bf0c09 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d184d745e54d4f629f9b89868405f29d" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 42143 } } peers { permanent_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 33025 } } peers { permanent_uuid: "c7f16b4829bd44a9952c892f26d8e8c9" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 39059 } }
I20260811 06:29:35.046864 32524 catalog_manager.cc:5495] T 6f9ef3df97c74376bce188bdf6bf0c09 P 47f4dcc280824ca09b9c04fe9ee5c3a2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 47f4dcc280824ca09b9c04fe9ee5c3a2 (127.28.239.194). New cstate: current_term: 1 leader_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d184d745e54d4f629f9b89868405f29d" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 42143 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 33025 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7f16b4829bd44a9952c892f26d8e8c9" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 39059 } health_report { overall_health: UNKNOWN } } }
Aug 11 06:29:35 dist-test-slave-gkw7 krb5kdc[32451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429772, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.254@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:29:35.103192 655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:35.088285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50294 (local address 127.28.239.254:46267)
0811 06:29:35.088472 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:35.088476 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:35.089187 (+ 711us) server_negotiation.cc:407] Connection header received
0811 06:29:35.089440 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:35.089450 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:35.089508 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:35.089649 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:35.090886 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:35.091720 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:35.093034 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:35.093650 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:35.093800 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:35.097506 (+ 3706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:29:35.097511 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:29:35.097514 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:29:35.097598 (+ 84us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:29:35.100286 (+ 2688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:35.101046 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:35.101059 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:35.101062 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:35.101185 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:35.101610 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:35.101617 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:35.101620 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:35.101919 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:29:35.102068 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:35.102521 (+ 453us) server_negotiation.cc:299] Negotiation successful
0811 06:29:35.102876 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":136,"server-negotiator.queue_time_us":67}
I20260811 06:29:35.103241 883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:35.088434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:46267 (local address 127.0.0.1:50294)
0811 06:29:35.089062 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:35.089079 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 06:29:35.089299 (+ 220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:29:35.089682 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:29:35.089686 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:29:35.090443 (+ 757us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:29:35.090716 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:35.090726 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:35.091869 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:35.091876 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:35.092876 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:35.092889 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:35.093897 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:35.093909 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:35.094106 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:35.094816 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:29:35.094837 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:29:35.097312 (+ 2475us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:29:35.100425 (+ 3113us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:29:35.100440 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:29:35.100449 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:29:35.100917 (+ 468us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:29:35.101327 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:29:35.101331 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:29:35.101335 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:29:35.101474 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:29:35.102060 (+ 586us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:29:35.102068 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:29:35.102244 (+ 176us) client_negotiation.cc:769] Sending connection context
0811 06:29:35.102420 (+ 176us) client_negotiation.cc:240] Negotiation successful
0811 06:29:35.102728 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":336,"threads_started":1}
I20260811 06:29:35.108721 32524 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.28.239.254@KRBTEST.COM'} at 127.0.0.1:50294:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 06:29:35.109920 32524 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:29:35.117578 507 tablet_service.cc:1467] Processing CreateTablet for tablet b8b1088a218b4fc58f0e100e05acb2a3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f727c9593bd84fee969caccdeb9d2e8d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 06:29:35.117467 32756 tablet_service.cc:1467] Processing CreateTablet for tablet b8b1088a218b4fc58f0e100e05acb2a3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f727c9593bd84fee969caccdeb9d2e8d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 06:29:35.117832 723 tablet_service.cc:1467] Processing CreateTablet for tablet b8b1088a218b4fc58f0e100e05acb2a3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f727c9593bd84fee969caccdeb9d2e8d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 06:29:35.118645 723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b8b1088a218b4fc58f0e100e05acb2a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:29:35.118660 32756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b8b1088a218b4fc58f0e100e05acb2a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:29:35.118674 507 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b8b1088a218b4fc58f0e100e05acb2a3. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 06:29:35.123335 402 tablet.cc:2369] T 6f9ef3df97c74376bce188bdf6bf0c09 P c7f16b4829bd44a9952c892f26d8e8c9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:29:35.124727 855 tablet_bootstrap.cc:492] T b8b1088a218b4fc58f0e100e05acb2a3 P d184d745e54d4f629f9b89868405f29d: Bootstrap starting.
I20260811 06:29:35.126305 855 tablet_bootstrap.cc:654] T b8b1088a218b4fc58f0e100e05acb2a3 P d184d745e54d4f629f9b89868405f29d: Neither blocks nor log segments found. Creating new log.
I20260811 06:29:35.126972 862 tablet_bootstrap.cc:492] T b8b1088a218b4fc58f0e100e05acb2a3 P c7f16b4829bd44a9952c892f26d8e8c9: Bootstrap starting.
I20260811 06:29:35.127934 854 tablet_bootstrap.cc:492] T b8b1088a218b4fc58f0e100e05acb2a3 P 47f4dcc280824ca09b9c04fe9ee5c3a2: Bootstrap starting.
I20260811 06:29:35.128743 862 tablet_bootstrap.cc:654] T b8b1088a218b4fc58f0e100e05acb2a3 P c7f16b4829bd44a9952c892f26d8e8c9: Neither blocks nor log segments found. Creating new log.
I20260811 06:29:35.129752 854 tablet_bootstrap.cc:654] T b8b1088a218b4fc58f0e100e05acb2a3 P 47f4dcc280824ca09b9c04fe9ee5c3a2: Neither blocks nor log segments found. Creating new log.
I20260811 06:29:35.130609 855 tablet_bootstrap.cc:492] T b8b1088a218b4fc58f0e100e05acb2a3 P d184d745e54d4f629f9b89868405f29d: No bootstrap required, opened a new log
I20260811 06:29:35.130805 855 ts_tablet_manager.cc:1397] T b8b1088a218b4fc58f0e100e05acb2a3 P d184d745e54d4f629f9b89868405f29d: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260811 06:29:35.131662 862 tablet_bootstrap.cc:492] T b8b1088a218b4fc58f0e100e05acb2a3 P c7f16b4829bd44a9952c892f26d8e8c9: No bootstrap required, opened a new log
I20260811 06:29:35.131592 855 raft_consensus.cc:348] T b8b1088a218b4fc58f0e100e05acb2a3 P d184d745e54d4f629f9b89868405f29d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7f16b4829bd44a9952c892f26d8e8c9" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 39059 } } peers { permanent_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 33025 } } peers { permanent_uuid: "d184d745e54d4f629f9b89868405f29d" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 42143 } }
I20260811 06:29:35.131855 862 ts_tablet_manager.cc:1397] T b8b1088a218b4fc58f0e100e05acb2a3 P c7f16b4829bd44a9952c892f26d8e8c9: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 06:29:35.131927 855 raft_consensus.cc:374] T b8b1088a218b4fc58f0e100e05acb2a3 P d184d745e54d4f629f9b89868405f29d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:29:35.132042 855 raft_consensus.cc:729] T b8b1088a218b4fc58f0e100e05acb2a3 P d184d745e54d4f629f9b89868405f29d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d184d745e54d4f629f9b89868405f29d, State: Initialized, Role: FOLLOWER
I20260811 06:29:35.132325 855 consensus_queue.cc:260] T b8b1088a218b4fc58f0e100e05acb2a3 P d184d745e54d4f629f9b89868405f29d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7f16b4829bd44a9952c892f26d8e8c9" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 39059 } } peers { permanent_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 33025 } } peers { permanent_uuid: "d184d745e54d4f629f9b89868405f29d" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 42143 } }
I20260811 06:29:35.132726 862 raft_consensus.cc:348] T b8b1088a218b4fc58f0e100e05acb2a3 P c7f16b4829bd44a9952c892f26d8e8c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7f16b4829bd44a9952c892f26d8e8c9" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 39059 } } peers { permanent_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 33025 } } peers { permanent_uuid: "d184d745e54d4f629f9b89868405f29d" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 42143 } }
I20260811 06:29:35.132977 862 raft_consensus.cc:374] T b8b1088a218b4fc58f0e100e05acb2a3 P c7f16b4829bd44a9952c892f26d8e8c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:29:35.133087 862 raft_consensus.cc:729] T b8b1088a218b4fc58f0e100e05acb2a3 P c7f16b4829bd44a9952c892f26d8e8c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7f16b4829bd44a9952c892f26d8e8c9, State: Initialized, Role: FOLLOWER
I20260811 06:29:35.132855 867 tablet_replica.cc:439] T b8b1088a218b4fc58f0e100e05acb2a3 P d184d745e54d4f629f9b89868405f29d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7f16b4829bd44a9952c892f26d8e8c9" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 39059 } } peers { permanent_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 33025 } } peers { permanent_uuid: "d184d745e54d4f629f9b89868405f29d" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 42143 } } }
I20260811 06:29:35.133230 854 tablet_bootstrap.cc:492] T b8b1088a218b4fc58f0e100e05acb2a3 P 47f4dcc280824ca09b9c04fe9ee5c3a2: No bootstrap required, opened a new log
I20260811 06:29:35.133275 867 tablet_replica.cc:442] T b8b1088a218b4fc58f0e100e05acb2a3 P d184d745e54d4f629f9b89868405f29d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 06:29:35.133376 854 ts_tablet_manager.cc:1397] T b8b1088a218b4fc58f0e100e05acb2a3 P 47f4dcc280824ca09b9c04fe9ee5c3a2: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.003s
I20260811 06:29:35.133338 862 consensus_queue.cc:260] T b8b1088a218b4fc58f0e100e05acb2a3 P c7f16b4829bd44a9952c892f26d8e8c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7f16b4829bd44a9952c892f26d8e8c9" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 39059 } } peers { permanent_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 33025 } } peers { permanent_uuid: "d184d745e54d4f629f9b89868405f29d" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 42143 } }
I20260811 06:29:35.133957 862 ts_tablet_manager.cc:1428] T b8b1088a218b4fc58f0e100e05acb2a3 P c7f16b4829bd44a9952c892f26d8e8c9: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 06:29:35.134240 854 raft_consensus.cc:348] T b8b1088a218b4fc58f0e100e05acb2a3 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7f16b4829bd44a9952c892f26d8e8c9" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 39059 } } peers { permanent_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 33025 } } peers { permanent_uuid: "d184d745e54d4f629f9b89868405f29d" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 42143 } }
I20260811 06:29:35.134479 854 raft_consensus.cc:374] T b8b1088a218b4fc58f0e100e05acb2a3 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:29:35.134577 854 raft_consensus.cc:729] T b8b1088a218b4fc58f0e100e05acb2a3 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47f4dcc280824ca09b9c04fe9ee5c3a2, State: Initialized, Role: FOLLOWER
I20260811 06:29:35.134744 855 ts_tablet_manager.cc:1428] T b8b1088a218b4fc58f0e100e05acb2a3 P d184d745e54d4f629f9b89868405f29d: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260811 06:29:35.134806 854 consensus_queue.cc:260] T b8b1088a218b4fc58f0e100e05acb2a3 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7f16b4829bd44a9952c892f26d8e8c9" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 39059 } } peers { permanent_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 33025 } } peers { permanent_uuid: "d184d745e54d4f629f9b89868405f29d" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 42143 } }
I20260811 06:29:35.135318 866 tablet_replica.cc:439] T b8b1088a218b4fc58f0e100e05acb2a3 P 47f4dcc280824ca09b9c04fe9ee5c3a2: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7f16b4829bd44a9952c892f26d8e8c9" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 39059 } } peers { permanent_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 33025 } } peers { permanent_uuid: "d184d745e54d4f629f9b89868405f29d" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 42143 } } }
I20260811 06:29:35.135213 868 tablet_replica.cc:439] T b8b1088a218b4fc58f0e100e05acb2a3 P c7f16b4829bd44a9952c892f26d8e8c9: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7f16b4829bd44a9952c892f26d8e8c9" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 39059 } } peers { permanent_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 33025 } } peers { permanent_uuid: "d184d745e54d4f629f9b89868405f29d" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 42143 } } }
I20260811 06:29:35.135613 866 tablet_replica.cc:442] T b8b1088a218b4fc58f0e100e05acb2a3 P 47f4dcc280824ca09b9c04fe9ee5c3a2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 06:29:35.135624 868 tablet_replica.cc:442] T b8b1088a218b4fc58f0e100e05acb2a3 P c7f16b4829bd44a9952c892f26d8e8c9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 06:29:35.136188 854 ts_tablet_manager.cc:1428] T b8b1088a218b4fc58f0e100e05acb2a3 P 47f4dcc280824ca09b9c04fe9ee5c3a2: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
W20260811 06:29:35.166064 619 tablet.cc:2369] T 6f9ef3df97c74376bce188bdf6bf0c09 P 47f4dcc280824ca09b9c04fe9ee5c3a2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:29:35.394222 868 raft_consensus.cc:482] T b8b1088a218b4fc58f0e100e05acb2a3 P c7f16b4829bd44a9952c892f26d8e8c9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:29:35.394537 868 raft_consensus.cc:504] T b8b1088a218b4fc58f0e100e05acb2a3 P c7f16b4829bd44a9952c892f26d8e8c9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7f16b4829bd44a9952c892f26d8e8c9" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 39059 } } peers { permanent_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 33025 } } peers { permanent_uuid: "d184d745e54d4f629f9b89868405f29d" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 42143 } }
I20260811 06:29:35.396517 868 leader_election.cc:288] T b8b1088a218b4fc58f0e100e05acb2a3 P c7f16b4829bd44a9952c892f26d8e8c9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 47f4dcc280824ca09b9c04fe9ee5c3a2 (127.28.239.194:33025), d184d745e54d4f629f9b89868405f29d (127.28.239.195:42143)
I20260811 06:29:35.397065 866 raft_consensus.cc:482] T b8b1088a218b4fc58f0e100e05acb2a3 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:29:35.397290 866 raft_consensus.cc:504] T b8b1088a218b4fc58f0e100e05acb2a3 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7f16b4829bd44a9952c892f26d8e8c9" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 39059 } } peers { permanent_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 33025 } } peers { permanent_uuid: "d184d745e54d4f629f9b89868405f29d" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 42143 } }
I20260811 06:29:35.398262 866 leader_election.cc:288] T b8b1088a218b4fc58f0e100e05acb2a3 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c7f16b4829bd44a9952c892f26d8e8c9 (127.28.239.193:39059), d184d745e54d4f629f9b89868405f29d (127.28.239.195:42143)
I20260811 06:29:35.399323 321 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8b1088a218b4fc58f0e100e05acb2a3" candidate_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7f16b4829bd44a9952c892f26d8e8c9" is_pre_election: true
I20260811 06:29:35.399659 321 raft_consensus.cc:2454] T b8b1088a218b4fc58f0e100e05acb2a3 P c7f16b4829bd44a9952c892f26d8e8c9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 47f4dcc280824ca09b9c04fe9ee5c3a2 in term 0.
I20260811 06:29:35.400413 762 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8b1088a218b4fc58f0e100e05acb2a3" candidate_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d184d745e54d4f629f9b89868405f29d" is_pre_election: true
I20260811 06:29:35.400627 431 leader_election.cc:302] T b8b1088a218b4fc58f0e100e05acb2a3 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47f4dcc280824ca09b9c04fe9ee5c3a2, c7f16b4829bd44a9952c892f26d8e8c9; no voters:
I20260811 06:29:35.400736 762 raft_consensus.cc:2454] T b8b1088a218b4fc58f0e100e05acb2a3 P d184d745e54d4f629f9b89868405f29d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 47f4dcc280824ca09b9c04fe9ee5c3a2 in term 0.
I20260811 06:29:35.401033 866 raft_consensus.cc:2781] T b8b1088a218b4fc58f0e100e05acb2a3 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:29:35.401142 866 raft_consensus.cc:482] T b8b1088a218b4fc58f0e100e05acb2a3 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:29:35.401284 866 raft_consensus.cc:3037] T b8b1088a218b4fc58f0e100e05acb2a3 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:29:35.405467 866 raft_consensus.cc:504] T b8b1088a218b4fc58f0e100e05acb2a3 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7f16b4829bd44a9952c892f26d8e8c9" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 39059 } } peers { permanent_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 33025 } } peers { permanent_uuid: "d184d745e54d4f629f9b89868405f29d" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 42143 } }
I20260811 06:29:35.407291 321 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8b1088a218b4fc58f0e100e05acb2a3" candidate_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7f16b4829bd44a9952c892f26d8e8c9"
I20260811 06:29:35.407621 321 raft_consensus.cc:3037] T b8b1088a218b4fc58f0e100e05acb2a3 P c7f16b4829bd44a9952c892f26d8e8c9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:29:35.411123 866 leader_election.cc:288] T b8b1088a218b4fc58f0e100e05acb2a3 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [CANDIDATE]: Term 1 election: Requested vote from peers c7f16b4829bd44a9952c892f26d8e8c9 (127.28.239.193:39059), d184d745e54d4f629f9b89868405f29d (127.28.239.195:42143)
I20260811 06:29:35.411747 762 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8b1088a218b4fc58f0e100e05acb2a3" candidate_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d184d745e54d4f629f9b89868405f29d"
I20260811 06:29:35.412027 762 raft_consensus.cc:3037] T b8b1088a218b4fc58f0e100e05acb2a3 P d184d745e54d4f629f9b89868405f29d [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:29:35.412714 321 raft_consensus.cc:2454] T b8b1088a218b4fc58f0e100e05acb2a3 P c7f16b4829bd44a9952c892f26d8e8c9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47f4dcc280824ca09b9c04fe9ee5c3a2 in term 1.
I20260811 06:29:35.415299 762 raft_consensus.cc:2454] T b8b1088a218b4fc58f0e100e05acb2a3 P d184d745e54d4f629f9b89868405f29d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47f4dcc280824ca09b9c04fe9ee5c3a2 in term 1.
I20260811 06:29:35.415633 431 leader_election.cc:302] T b8b1088a218b4fc58f0e100e05acb2a3 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47f4dcc280824ca09b9c04fe9ee5c3a2, c7f16b4829bd44a9952c892f26d8e8c9; no voters:
I20260811 06:29:35.415980 866 raft_consensus.cc:2781] T b8b1088a218b4fc58f0e100e05acb2a3 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:29:35.416175 866 raft_consensus.cc:686] T b8b1088a218b4fc58f0e100e05acb2a3 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [term 1 LEADER]: Becoming Leader. State: Replica: 47f4dcc280824ca09b9c04fe9ee5c3a2, State: Running, Role: LEADER
I20260811 06:29:35.416465 866 consensus_queue.cc:237] T b8b1088a218b4fc58f0e100e05acb2a3 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7f16b4829bd44a9952c892f26d8e8c9" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 39059 } } peers { permanent_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 33025 } } peers { permanent_uuid: "d184d745e54d4f629f9b89868405f29d" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 42143 } }
I20260811 06:29:35.418352 871 tablet_replica.cc:439] T b8b1088a218b4fc58f0e100e05acb2a3 P 47f4dcc280824ca09b9c04fe9ee5c3a2: TxnStatusTablet state changed. Reason: New leader 47f4dcc280824ca09b9c04fe9ee5c3a2. Latest consensus state: current_term: 1 leader_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7f16b4829bd44a9952c892f26d8e8c9" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 39059 } } peers { permanent_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 33025 } } peers { permanent_uuid: "d184d745e54d4f629f9b89868405f29d" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 42143 } } }
I20260811 06:29:35.418794 871 tablet_replica.cc:442] T b8b1088a218b4fc58f0e100e05acb2a3 P 47f4dcc280824ca09b9c04fe9ee5c3a2: This TxnStatusTablet replica's current role is: LEADER
I20260811 06:29:35.419301 888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:35.396889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.194:33025 (local address 127.28.239.193:39861)
0811 06:29:35.410172 (+ 13283us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:35.410203 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0811 06:29:35.410393 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:29:35.412164 (+ 1771us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:29:35.412170 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:29:35.412198 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:29:35.412678 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:35.412699 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:35.414139 (+ 1440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:35.414144 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:35.416430 (+ 2286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:35.416444 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:35.418367 (+ 1923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:35.418377 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:35.418570 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:35.418626 (+ 56us) client_negotiation.cc:769] Sending connection context
0811 06:29:35.418730 (+ 104us) client_negotiation.cc:240] Negotiation successful
0811 06:29:35.418885 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13066,"mutex_wait_us":1,"thread_start_us":13624,"threads_started":1}
I20260811 06:29:35.420202 890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:35.397071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.193:39861 (local address 127.28.239.194:33025)
0811 06:29:35.411705 (+ 14634us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:35.411711 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:35.411731 (+ 20us) server_negotiation.cc:407] Connection header received
0811 06:29:35.411808 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:35.411813 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:35.411895 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:35.411997 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:29:35.412936 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:35.413959 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:35.416628 (+ 2669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:35.417914 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:35.418115 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:35.418192 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:35.419312 (+ 1120us) server_negotiation.cc:299] Negotiation successful
0811 06:29:35.419665 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14472,"thread_start_us":14344,"threads_started":1}
I20260811 06:29:35.420382 850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:35.397228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.193:33407 (local address 127.28.239.195:42143)
0811 06:29:35.397447 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:35.397457 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:35.412565 (+ 15108us) server_negotiation.cc:407] Connection header received
0811 06:29:35.412857 (+ 292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:35.412863 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:35.412936 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:35.413008 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:29:35.413853 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:35.415038 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:35.417813 (+ 2775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:35.419122 (+ 1309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:35.419261 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:35.419360 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:35.419962 (+ 602us) server_negotiation.cc:299] Negotiation successful
0811 06:29:35.420114 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 06:29:35.421438 893 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 06:29:35.421845 893 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 06:29:35.421844 889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:35.396967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.195:42143 (local address 127.28.239.193:33407)
0811 06:29:35.412444 (+ 15477us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:35.412468 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 06:29:35.412680 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:29:35.413244 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:29:35.413248 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:29:35.413274 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:29:35.413647 (+ 373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:35.413659 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:35.415248 (+ 1589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:35.415254 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:35.417477 (+ 2223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:35.417498 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:35.419576 (+ 2078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:35.419583 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:35.419763 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:35.419794 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 06:29:35.420621 (+ 827us) client_negotiation.cc:240] Negotiation successful
0811 06:29:35.421404 (+ 783us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14919,"thread_start_us":14710,"threads_started":1}
I20260811 06:29:35.421689 548 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8b1088a218b4fc58f0e100e05acb2a3" candidate_uuid: "c7f16b4829bd44a9952c892f26d8e8c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" is_pre_election: true
I20260811 06:29:35.421029 32527 catalog_manager.cc:5495] T b8b1088a218b4fc58f0e100e05acb2a3 P 47f4dcc280824ca09b9c04fe9ee5c3a2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 47f4dcc280824ca09b9c04fe9ee5c3a2 (127.28.239.194). New cstate: current_term: 1 leader_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7f16b4829bd44a9952c892f26d8e8c9" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 39059 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 33025 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d184d745e54d4f629f9b89868405f29d" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 42143 } health_report { overall_health: UNKNOWN } } }
I20260811 06:29:35.422680 762 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8b1088a218b4fc58f0e100e05acb2a3" candidate_uuid: "c7f16b4829bd44a9952c892f26d8e8c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d184d745e54d4f629f9b89868405f29d" is_pre_election: true
I20260811 06:29:35.423182 762 raft_consensus.cc:2379] T b8b1088a218b4fc58f0e100e05acb2a3 P d184d745e54d4f629f9b89868405f29d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7f16b4829bd44a9952c892f26d8e8c9 in current term 1: Already voted for candidate 47f4dcc280824ca09b9c04fe9ee5c3a2 in this term.
I20260811 06:29:35.424275 32673 leader_election.cc:302] T b8b1088a218b4fc58f0e100e05acb2a3 P c7f16b4829bd44a9952c892f26d8e8c9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7f16b4829bd44a9952c892f26d8e8c9; no voters: 47f4dcc280824ca09b9c04fe9ee5c3a2, d184d745e54d4f629f9b89868405f29d
I20260811 06:29:35.425854 868 raft_consensus.cc:2726] T b8b1088a218b4fc58f0e100e05acb2a3 P c7f16b4829bd44a9952c892f26d8e8c9 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 06:29:35.512602 883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:35.506073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.194:33025 (local address 127.0.0.1:44840)
0811 06:29:35.506346 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:35.506379 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0811 06:29:35.506501 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:29:35.507086 (+ 585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:29:35.507089 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:29:35.507100 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 06:29:35.507276 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:35.507284 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:35.508798 (+ 1514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:35.508802 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:35.509901 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:35.509913 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:35.510900 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:35.510906 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:35.511070 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:35.511127 (+ 57us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 06:29:35.512190 (+ 1063us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 06:29:35.512206 (+ 16us) client_negotiation.cc:769] Sending connection context
0811 06:29:35.512262 (+ 56us) client_negotiation.cc:240] Negotiation successful
0811 06:29:35.512376 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260811 06:29:35.512691 890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:35.506261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44840 (local address 127.28.239.194:33025)
0811 06:29:35.506482 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:35.506490 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:35.506512 (+ 22us) server_negotiation.cc:407] Connection header received
0811 06:29:35.506738 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:35.506744 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:35.506859 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:35.507029 (+ 170us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:29:35.507418 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:35.508652 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:35.510038 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:35.510721 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:35.510849 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:35.511327 (+ 478us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:29:35.512077 (+ 750us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:29:35.512229 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:35.512286 (+ 57us) server_negotiation.cc:299] Negotiation successful
0811 06:29:35.512377 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 06:29:35.521797 760 raft_consensus.cc:1261] T b8b1088a218b4fc58f0e100e05acb2a3 P d184d745e54d4f629f9b89868405f29d [term 1 FOLLOWER]: Refusing update from remote peer 47f4dcc280824ca09b9c04fe9ee5c3a2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:29:35.521944 321 raft_consensus.cc:1261] T b8b1088a218b4fc58f0e100e05acb2a3 P c7f16b4829bd44a9952c892f26d8e8c9 [term 1 FOLLOWER]: Refusing update from remote peer 47f4dcc280824ca09b9c04fe9ee5c3a2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:29:35.522775 866 consensus_queue.cc:1035] T b8b1088a218b4fc58f0e100e05acb2a3 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d184d745e54d4f629f9b89868405f29d" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 42143 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:29:35.522733 898 consensus_queue.cc:1035] T 6f9ef3df97c74376bce188bdf6bf0c09 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d184d745e54d4f629f9b89868405f29d" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 42143 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:29:35.523244 871 consensus_queue.cc:1035] T b8b1088a218b4fc58f0e100e05acb2a3 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7f16b4829bd44a9952c892f26d8e8c9" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 39059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:29:35.536463 871 consensus_queue.cc:1035] T 6f9ef3df97c74376bce188bdf6bf0c09 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7f16b4829bd44a9952c892f26d8e8c9" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 39059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:29:35.547868 868 tablet_replica.cc:439] T b8b1088a218b4fc58f0e100e05acb2a3 P c7f16b4829bd44a9952c892f26d8e8c9: TxnStatusTablet state changed. Reason: New leader 47f4dcc280824ca09b9c04fe9ee5c3a2. Latest consensus state: current_term: 1 leader_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7f16b4829bd44a9952c892f26d8e8c9" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 39059 } } peers { permanent_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 33025 } } peers { permanent_uuid: "d184d745e54d4f629f9b89868405f29d" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 42143 } } }
I20260811 06:29:35.548235 868 tablet_replica.cc:442] T b8b1088a218b4fc58f0e100e05acb2a3 P c7f16b4829bd44a9952c892f26d8e8c9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 06:29:35.553331 897 mvcc.cc:205] Tried to move back new op lower bound from 7317216360524894208 to 7317216360108556288. Current Snapshot: MvccSnapshot[applied={T|T < 7317216360524894208}]
I20260811 06:29:35.554605 902 mvcc.cc:205] Tried to move back new op lower bound from 7317216360524894208 to 7317216360108556288. Current Snapshot: MvccSnapshot[applied={T|T < 7317216360524894208}]
I20260811 06:29:35.557446 868 tablet_replica.cc:439] T b8b1088a218b4fc58f0e100e05acb2a3 P c7f16b4829bd44a9952c892f26d8e8c9: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7f16b4829bd44a9952c892f26d8e8c9" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 39059 } } peers { permanent_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 33025 } } peers { permanent_uuid: "d184d745e54d4f629f9b89868405f29d" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 42143 } } }
I20260811 06:29:35.557766 868 tablet_replica.cc:442] T b8b1088a218b4fc58f0e100e05acb2a3 P c7f16b4829bd44a9952c892f26d8e8c9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 06:29:35.559532 867 tablet_replica.cc:439] T b8b1088a218b4fc58f0e100e05acb2a3 P d184d745e54d4f629f9b89868405f29d: TxnStatusTablet state changed. Reason: New leader 47f4dcc280824ca09b9c04fe9ee5c3a2. Latest consensus state: current_term: 1 leader_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7f16b4829bd44a9952c892f26d8e8c9" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 39059 } } peers { permanent_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 33025 } } peers { permanent_uuid: "d184d745e54d4f629f9b89868405f29d" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 42143 } } }
I20260811 06:29:35.559860 867 tablet_replica.cc:442] T b8b1088a218b4fc58f0e100e05acb2a3 P d184d745e54d4f629f9b89868405f29d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 06:29:35.566088 898 tablet_replica.cc:439] T b8b1088a218b4fc58f0e100e05acb2a3 P 47f4dcc280824ca09b9c04fe9ee5c3a2: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7f16b4829bd44a9952c892f26d8e8c9" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 39059 } } peers { permanent_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 33025 } } peers { permanent_uuid: "d184d745e54d4f629f9b89868405f29d" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 42143 } } }
I20260811 06:29:35.566179 871 tablet_replica.cc:439] T b8b1088a218b4fc58f0e100e05acb2a3 P 47f4dcc280824ca09b9c04fe9ee5c3a2: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7f16b4829bd44a9952c892f26d8e8c9" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 39059 } } peers { permanent_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 33025 } } peers { permanent_uuid: "d184d745e54d4f629f9b89868405f29d" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 42143 } } }
I20260811 06:29:35.566386 898 tablet_replica.cc:442] T b8b1088a218b4fc58f0e100e05acb2a3 P 47f4dcc280824ca09b9c04fe9ee5c3a2: This TxnStatusTablet replica's current role is: LEADER
I20260811 06:29:35.566439 871 tablet_replica.cc:442] T b8b1088a218b4fc58f0e100e05acb2a3 P 47f4dcc280824ca09b9c04fe9ee5c3a2: This TxnStatusTablet replica's current role is: LEADER
I20260811 06:29:35.568310 867 tablet_replica.cc:439] T b8b1088a218b4fc58f0e100e05acb2a3 P d184d745e54d4f629f9b89868405f29d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7f16b4829bd44a9952c892f26d8e8c9" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 39059 } } peers { permanent_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 33025 } } peers { permanent_uuid: "d184d745e54d4f629f9b89868405f29d" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 42143 } } }
I20260811 06:29:35.568612 867 tablet_replica.cc:442] T b8b1088a218b4fc58f0e100e05acb2a3 P d184d745e54d4f629f9b89868405f29d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 06:29:35.583568 890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:35.576993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44842 (local address 127.28.239.194:33025)
0811 06:29:35.577190 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:35.577197 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:35.577596 (+ 399us) server_negotiation.cc:407] Connection header received
0811 06:29:35.577830 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:35.577858 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:35.577962 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:35.578078 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:29:35.578473 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:35.579918 (+ 1445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:35.581411 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:35.582229 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:35.582372 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:35.582723 (+ 351us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:29:35.582887 (+ 164us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:29:35.582993 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:35.583115 (+ 122us) server_negotiation.cc:299] Negotiation successful
0811 06:29:35.583208 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 06:29:35.601855 923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:35.595006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.194:33025 (local address 127.28.239.194:46327)
0811 06:29:35.595833 (+ 827us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:35.595849 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:29:35.595969 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:29:35.596314 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:29:35.596320 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:29:35.596330 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 06:29:35.596582 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:35.596589 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:35.598214 (+ 1625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:35.598218 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:35.599385 (+ 1167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:35.599396 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:35.600515 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:35.600521 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:35.600690 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:35.600754 (+ 64us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 06:29:35.601263 (+ 509us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 06:29:35.601272 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 06:29:35.601338 (+ 66us) client_negotiation.cc:240] Negotiation successful
0811 06:29:35.601445 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":672,"thread_start_us":400,"threads_started":1}
I20260811 06:29:35.601884 890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:35.595146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.194:46327 (local address 127.28.239.194:33025)
0811 06:29:35.595348 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:35.595369 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:35.595936 (+ 567us) server_negotiation.cc:407] Connection header received
0811 06:29:35.596095 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:35.596101 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:35.596185 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:35.596302 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:29:35.596729 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:35.598065 (+ 1336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:35.599536 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:35.600323 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:35.600475 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:35.600970 (+ 495us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:29:35.601152 (+ 182us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:29:35.601253 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:35.601396 (+ 143us) server_negotiation.cc:299] Negotiation successful
0811 06:29:35.601486 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 06:29:35.695485 32527 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50278:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:29:35.695801 32527 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50278:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:29:35.700074 32527 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 15e16a76dba440648ce59c6d9f4b7a8c: Sending DeleteTablet for 3 replicas of tablet 6f9ef3df97c74376bce188bdf6bf0c09
I20260811 06:29:35.701522 506 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f9ef3df97c74376bce188bdf6bf0c09 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:29:35 UTC) from {username='kudu', principal='kudu/127.28.239.254@KRBTEST.COM'} at 127.0.0.1:44830
I20260811 06:29:35.701512 723 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f9ef3df97c74376bce188bdf6bf0c09 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:29:35 UTC) from {username='kudu', principal='kudu/127.28.239.254@KRBTEST.COM'} at 127.0.0.1:35512
I20260811 06:29:35.702512 32756 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f9ef3df97c74376bce188bdf6bf0c09 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:29:35 UTC) from {username='kudu', principal='kudu/127.28.239.254@KRBTEST.COM'} at 127.0.0.1:34340
I20260811 06:29:35.703794 32527 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50278:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:29:35.704505 32527 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:29:35.708669 935 tablet_replica.cc:330] T 6f9ef3df97c74376bce188bdf6bf0c09 P c7f16b4829bd44a9952c892f26d8e8c9: stopping tablet replica
I20260811 06:29:35.709239 935 raft_consensus.cc:2229] T 6f9ef3df97c74376bce188bdf6bf0c09 P c7f16b4829bd44a9952c892f26d8e8c9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:29:35.709710 935 raft_consensus.cc:2258] T 6f9ef3df97c74376bce188bdf6bf0c09 P c7f16b4829bd44a9952c892f26d8e8c9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:29:35.713922 721 tablet_service.cc:1467] Processing CreateTablet for tablet 208454ead35b4463844e580cea7ff138 (DEFAULT_TABLE table=test-table [id=006bbd0b31164ff5810fc760d90cc93b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:29:35.714171 935 ts_tablet_manager.cc:1905] T 6f9ef3df97c74376bce188bdf6bf0c09 P c7f16b4829bd44a9952c892f26d8e8c9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:29:35.714504 721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 208454ead35b4463844e580cea7ff138. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:29:35.717940 930 tablet_replica.cc:330] T 6f9ef3df97c74376bce188bdf6bf0c09 P d184d745e54d4f629f9b89868405f29d: stopping tablet replica
I20260811 06:29:35.718065 32756 tablet_service.cc:1467] Processing CreateTablet for tablet 208454ead35b4463844e580cea7ff138 (DEFAULT_TABLE table=test-table [id=006bbd0b31164ff5810fc760d90cc93b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:29:35.718591 32756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 208454ead35b4463844e580cea7ff138. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:29:35.718560 930 raft_consensus.cc:2229] T 6f9ef3df97c74376bce188bdf6bf0c09 P d184d745e54d4f629f9b89868405f29d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:29:35.719041 930 raft_consensus.cc:2258] T 6f9ef3df97c74376bce188bdf6bf0c09 P d184d745e54d4f629f9b89868405f29d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:29:35.721056 930 ts_tablet_manager.cc:1905] T 6f9ef3df97c74376bce188bdf6bf0c09 P d184d745e54d4f629f9b89868405f29d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:29:35.723286 933 tablet_replica.cc:330] T 6f9ef3df97c74376bce188bdf6bf0c09 P 47f4dcc280824ca09b9c04fe9ee5c3a2: stopping tablet replica
I20260811 06:29:35.723728 933 raft_consensus.cc:2229] T 6f9ef3df97c74376bce188bdf6bf0c09 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:29:35.724504 933 raft_consensus.cc:2258] T 6f9ef3df97c74376bce188bdf6bf0c09 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:29:35.727197 936 tablet_bootstrap.cc:492] T 208454ead35b4463844e580cea7ff138 P d184d745e54d4f629f9b89868405f29d: Bootstrap starting.
I20260811 06:29:35.729444 935 ts_tablet_manager.cc:1918] T 6f9ef3df97c74376bce188bdf6bf0c09 P c7f16b4829bd44a9952c892f26d8e8c9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 06:29:35.729687 935 log.cc:1196] T 6f9ef3df97c74376bce188bdf6bf0c09 P c7f16b4829bd44a9952c892f26d8e8c9: Deleting WAL directory at /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/minicluster-data/ts-0/wal/wals/6f9ef3df97c74376bce188bdf6bf0c09
I20260811 06:29:35.730337 935 ts_tablet_manager.cc:1939] T 6f9ef3df97c74376bce188bdf6bf0c09 P c7f16b4829bd44a9952c892f26d8e8c9: Deleting consensus metadata
I20260811 06:29:35.731086 506 tablet_service.cc:1467] Processing CreateTablet for tablet 208454ead35b4463844e580cea7ff138 (DEFAULT_TABLE table=test-table [id=006bbd0b31164ff5810fc760d90cc93b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:29:35.731674 506 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 208454ead35b4463844e580cea7ff138. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:29:35.732375 936 tablet_bootstrap.cc:654] T 208454ead35b4463844e580cea7ff138 P d184d745e54d4f629f9b89868405f29d: Neither blocks nor log segments found. Creating new log.
I20260811 06:29:35.732402 32497 catalog_manager.cc:4841] TS c7f16b4829bd44a9952c892f26d8e8c9 (127.28.239.193:39059): tablet 6f9ef3df97c74376bce188bdf6bf0c09 (table test-table [id=70e2a79884c2478899214d69e382a7d4]) successfully deleted
I20260811 06:29:35.730062 933 ts_tablet_manager.cc:1905] T 6f9ef3df97c74376bce188bdf6bf0c09 P 47f4dcc280824ca09b9c04fe9ee5c3a2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:29:35.733645 930 ts_tablet_manager.cc:1918] T 6f9ef3df97c74376bce188bdf6bf0c09 P d184d745e54d4f629f9b89868405f29d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 06:29:35.733881 930 log.cc:1196] T 6f9ef3df97c74376bce188bdf6bf0c09 P d184d745e54d4f629f9b89868405f29d: Deleting WAL directory at /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/minicluster-data/ts-2/wal/wals/6f9ef3df97c74376bce188bdf6bf0c09
I20260811 06:29:35.734655 930 ts_tablet_manager.cc:1939] T 6f9ef3df97c74376bce188bdf6bf0c09 P d184d745e54d4f629f9b89868405f29d: Deleting consensus metadata
I20260811 06:29:35.736618 32497 catalog_manager.cc:4841] TS d184d745e54d4f629f9b89868405f29d (127.28.239.195:42143): tablet 6f9ef3df97c74376bce188bdf6bf0c09 (table test-table [id=70e2a79884c2478899214d69e382a7d4]) successfully deleted
I20260811 06:29:35.743145 939 tablet_bootstrap.cc:492] T 208454ead35b4463844e580cea7ff138 P c7f16b4829bd44a9952c892f26d8e8c9: Bootstrap starting.
I20260811 06:29:35.746218 938 tablet_bootstrap.cc:492] T 208454ead35b4463844e580cea7ff138 P 47f4dcc280824ca09b9c04fe9ee5c3a2: Bootstrap starting.
I20260811 06:29:35.747742 939 tablet_bootstrap.cc:654] T 208454ead35b4463844e580cea7ff138 P c7f16b4829bd44a9952c892f26d8e8c9: Neither blocks nor log segments found. Creating new log.
I20260811 06:29:35.748739 933 ts_tablet_manager.cc:1918] T 6f9ef3df97c74376bce188bdf6bf0c09 P 47f4dcc280824ca09b9c04fe9ee5c3a2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 06:29:35.748813 938 tablet_bootstrap.cc:654] T 208454ead35b4463844e580cea7ff138 P 47f4dcc280824ca09b9c04fe9ee5c3a2: Neither blocks nor log segments found. Creating new log.
I20260811 06:29:35.748931 933 log.cc:1196] T 6f9ef3df97c74376bce188bdf6bf0c09 P 47f4dcc280824ca09b9c04fe9ee5c3a2: Deleting WAL directory at /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/minicluster-data/ts-1/wal/wals/6f9ef3df97c74376bce188bdf6bf0c09
I20260811 06:29:35.749665 933 ts_tablet_manager.cc:1939] T 6f9ef3df97c74376bce188bdf6bf0c09 P 47f4dcc280824ca09b9c04fe9ee5c3a2: Deleting consensus metadata
I20260811 06:29:35.751961 32501 catalog_manager.cc:4841] TS 47f4dcc280824ca09b9c04fe9ee5c3a2 (127.28.239.194:33025): tablet 6f9ef3df97c74376bce188bdf6bf0c09 (table test-table [id=70e2a79884c2478899214d69e382a7d4]) successfully deleted
I20260811 06:29:35.752960 936 tablet_bootstrap.cc:492] T 208454ead35b4463844e580cea7ff138 P d184d745e54d4f629f9b89868405f29d: No bootstrap required, opened a new log
I20260811 06:29:35.753173 936 ts_tablet_manager.cc:1397] T 208454ead35b4463844e580cea7ff138 P d184d745e54d4f629f9b89868405f29d: Time spent bootstrapping tablet: real 0.026s user 0.008s sys 0.008s
I20260811 06:29:35.753890 939 tablet_bootstrap.cc:492] T 208454ead35b4463844e580cea7ff138 P c7f16b4829bd44a9952c892f26d8e8c9: No bootstrap required, opened a new log
I20260811 06:29:35.754063 939 ts_tablet_manager.cc:1397] T 208454ead35b4463844e580cea7ff138 P c7f16b4829bd44a9952c892f26d8e8c9: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.005s
I20260811 06:29:35.754189 936 raft_consensus.cc:348] T 208454ead35b4463844e580cea7ff138 P d184d745e54d4f629f9b89868405f29d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7f16b4829bd44a9952c892f26d8e8c9" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 39059 } } peers { permanent_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 33025 } } peers { permanent_uuid: "d184d745e54d4f629f9b89868405f29d" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 42143 } }
I20260811 06:29:35.754474 936 raft_consensus.cc:374] T 208454ead35b4463844e580cea7ff138 P d184d745e54d4f629f9b89868405f29d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:29:35.754599 936 raft_consensus.cc:729] T 208454ead35b4463844e580cea7ff138 P d184d745e54d4f629f9b89868405f29d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d184d745e54d4f629f9b89868405f29d, State: Initialized, Role: FOLLOWER
I20260811 06:29:35.754897 939 raft_consensus.cc:348] T 208454ead35b4463844e580cea7ff138 P c7f16b4829bd44a9952c892f26d8e8c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7f16b4829bd44a9952c892f26d8e8c9" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 39059 } } peers { permanent_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 33025 } } peers { permanent_uuid: "d184d745e54d4f629f9b89868405f29d" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 42143 } }
I20260811 06:29:35.754935 936 consensus_queue.cc:260] T 208454ead35b4463844e580cea7ff138 P d184d745e54d4f629f9b89868405f29d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7f16b4829bd44a9952c892f26d8e8c9" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 39059 } } peers { permanent_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 33025 } } peers { permanent_uuid: "d184d745e54d4f629f9b89868405f29d" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 42143 } }
I20260811 06:29:35.755148 939 raft_consensus.cc:374] T 208454ead35b4463844e580cea7ff138 P c7f16b4829bd44a9952c892f26d8e8c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:29:35.755216 938 tablet_bootstrap.cc:492] T 208454ead35b4463844e580cea7ff138 P 47f4dcc280824ca09b9c04fe9ee5c3a2: No bootstrap required, opened a new log
I20260811 06:29:35.755270 939 raft_consensus.cc:729] T 208454ead35b4463844e580cea7ff138 P c7f16b4829bd44a9952c892f26d8e8c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7f16b4829bd44a9952c892f26d8e8c9, State: Initialized, Role: FOLLOWER
I20260811 06:29:35.755462 938 ts_tablet_manager.cc:1397] T 208454ead35b4463844e580cea7ff138 P 47f4dcc280824ca09b9c04fe9ee5c3a2: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.000s
I20260811 06:29:35.755558 939 consensus_queue.cc:260] T 208454ead35b4463844e580cea7ff138 P c7f16b4829bd44a9952c892f26d8e8c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7f16b4829bd44a9952c892f26d8e8c9" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 39059 } } peers { permanent_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 33025 } } peers { permanent_uuid: "d184d745e54d4f629f9b89868405f29d" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 42143 } }
I20260811 06:29:35.756389 938 raft_consensus.cc:348] T 208454ead35b4463844e580cea7ff138 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7f16b4829bd44a9952c892f26d8e8c9" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 39059 } } peers { permanent_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 33025 } } peers { permanent_uuid: "d184d745e54d4f629f9b89868405f29d" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 42143 } }
I20260811 06:29:35.756716 938 raft_consensus.cc:374] T 208454ead35b4463844e580cea7ff138 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:29:35.756880 938 raft_consensus.cc:729] T 208454ead35b4463844e580cea7ff138 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47f4dcc280824ca09b9c04fe9ee5c3a2, State: Initialized, Role: FOLLOWER
I20260811 06:29:35.757153 939 ts_tablet_manager.cc:1428] T 208454ead35b4463844e580cea7ff138 P c7f16b4829bd44a9952c892f26d8e8c9: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260811 06:29:35.757254 938 consensus_queue.cc:260] T 208454ead35b4463844e580cea7ff138 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7f16b4829bd44a9952c892f26d8e8c9" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 39059 } } peers { permanent_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 33025 } } peers { permanent_uuid: "d184d745e54d4f629f9b89868405f29d" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 42143 } }
I20260811 06:29:35.757949 938 ts_tablet_manager.cc:1428] T 208454ead35b4463844e580cea7ff138 P 47f4dcc280824ca09b9c04fe9ee5c3a2: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 06:29:35.759407 936 ts_tablet_manager.cc:1428] T 208454ead35b4463844e580cea7ff138 P d184d745e54d4f629f9b89868405f29d: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
I20260811 06:29:36.125829 871 raft_consensus.cc:482] T 208454ead35b4463844e580cea7ff138 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:29:36.126086 871 raft_consensus.cc:504] T 208454ead35b4463844e580cea7ff138 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7f16b4829bd44a9952c892f26d8e8c9" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 39059 } } peers { permanent_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 33025 } } peers { permanent_uuid: "d184d745e54d4f629f9b89868405f29d" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 42143 } }
I20260811 06:29:36.126977 871 leader_election.cc:288] T 208454ead35b4463844e580cea7ff138 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c7f16b4829bd44a9952c892f26d8e8c9 (127.28.239.193:39059), d184d745e54d4f629f9b89868405f29d (127.28.239.195:42143)
I20260811 06:29:36.127589 319 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "208454ead35b4463844e580cea7ff138" candidate_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7f16b4829bd44a9952c892f26d8e8c9" is_pre_election: true
I20260811 06:29:36.127768 760 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "208454ead35b4463844e580cea7ff138" candidate_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d184d745e54d4f629f9b89868405f29d" is_pre_election: true
I20260811 06:29:36.127888 319 raft_consensus.cc:2454] T 208454ead35b4463844e580cea7ff138 P c7f16b4829bd44a9952c892f26d8e8c9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 47f4dcc280824ca09b9c04fe9ee5c3a2 in term 0.
I20260811 06:29:36.128034 760 raft_consensus.cc:2454] T 208454ead35b4463844e580cea7ff138 P d184d745e54d4f629f9b89868405f29d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 47f4dcc280824ca09b9c04fe9ee5c3a2 in term 0.
I20260811 06:29:36.128448 431 leader_election.cc:302] T 208454ead35b4463844e580cea7ff138 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47f4dcc280824ca09b9c04fe9ee5c3a2, c7f16b4829bd44a9952c892f26d8e8c9; no voters:
I20260811 06:29:36.128746 871 raft_consensus.cc:2781] T 208454ead35b4463844e580cea7ff138 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:29:36.128908 871 raft_consensus.cc:482] T 208454ead35b4463844e580cea7ff138 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:29:36.129019 871 raft_consensus.cc:3037] T 208454ead35b4463844e580cea7ff138 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:29:36.132666 871 raft_consensus.cc:504] T 208454ead35b4463844e580cea7ff138 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7f16b4829bd44a9952c892f26d8e8c9" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 39059 } } peers { permanent_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 33025 } } peers { permanent_uuid: "d184d745e54d4f629f9b89868405f29d" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 42143 } }
I20260811 06:29:36.133410 871 leader_election.cc:288] T 208454ead35b4463844e580cea7ff138 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [CANDIDATE]: Term 1 election: Requested vote from peers c7f16b4829bd44a9952c892f26d8e8c9 (127.28.239.193:39059), d184d745e54d4f629f9b89868405f29d (127.28.239.195:42143)
I20260811 06:29:36.134021 319 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "208454ead35b4463844e580cea7ff138" candidate_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7f16b4829bd44a9952c892f26d8e8c9"
I20260811 06:29:36.134034 760 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "208454ead35b4463844e580cea7ff138" candidate_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d184d745e54d4f629f9b89868405f29d"
I20260811 06:29:36.134266 319 raft_consensus.cc:3037] T 208454ead35b4463844e580cea7ff138 P c7f16b4829bd44a9952c892f26d8e8c9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:29:36.134294 760 raft_consensus.cc:3037] T 208454ead35b4463844e580cea7ff138 P d184d745e54d4f629f9b89868405f29d [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:29:36.137310 760 raft_consensus.cc:2454] T 208454ead35b4463844e580cea7ff138 P d184d745e54d4f629f9b89868405f29d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47f4dcc280824ca09b9c04fe9ee5c3a2 in term 1.
I20260811 06:29:36.137326 319 raft_consensus.cc:2454] T 208454ead35b4463844e580cea7ff138 P c7f16b4829bd44a9952c892f26d8e8c9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47f4dcc280824ca09b9c04fe9ee5c3a2 in term 1.
I20260811 06:29:36.137807 431 leader_election.cc:302] T 208454ead35b4463844e580cea7ff138 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47f4dcc280824ca09b9c04fe9ee5c3a2, c7f16b4829bd44a9952c892f26d8e8c9; no voters:
I20260811 06:29:36.138190 871 raft_consensus.cc:2781] T 208454ead35b4463844e580cea7ff138 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:29:36.138378 871 raft_consensus.cc:686] T 208454ead35b4463844e580cea7ff138 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [term 1 LEADER]: Becoming Leader. State: Replica: 47f4dcc280824ca09b9c04fe9ee5c3a2, State: Running, Role: LEADER
I20260811 06:29:36.138809 871 consensus_queue.cc:237] T 208454ead35b4463844e580cea7ff138 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7f16b4829bd44a9952c892f26d8e8c9" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 39059 } } peers { permanent_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 33025 } } peers { permanent_uuid: "d184d745e54d4f629f9b89868405f29d" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 42143 } }
I20260811 06:29:36.142576 32523 catalog_manager.cc:5495] T 208454ead35b4463844e580cea7ff138 P 47f4dcc280824ca09b9c04fe9ee5c3a2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 47f4dcc280824ca09b9c04fe9ee5c3a2 (127.28.239.194). New cstate: current_term: 1 leader_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7f16b4829bd44a9952c892f26d8e8c9" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 39059 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 33025 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d184d745e54d4f629f9b89868405f29d" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 42143 } health_report { overall_health: UNKNOWN } } }
I20260811 06:29:36.218209 319 raft_consensus.cc:1261] T 208454ead35b4463844e580cea7ff138 P c7f16b4829bd44a9952c892f26d8e8c9 [term 1 FOLLOWER]: Refusing update from remote peer 47f4dcc280824ca09b9c04fe9ee5c3a2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:29:36.218222 760 raft_consensus.cc:1261] T 208454ead35b4463844e580cea7ff138 P d184d745e54d4f629f9b89868405f29d [term 1 FOLLOWER]: Refusing update from remote peer 47f4dcc280824ca09b9c04fe9ee5c3a2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:29:36.218806 955 consensus_queue.cc:1035] T 208454ead35b4463844e580cea7ff138 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7f16b4829bd44a9952c892f26d8e8c9" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 39059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:29:36.219189 924 consensus_queue.cc:1035] T 208454ead35b4463844e580cea7ff138 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d184d745e54d4f629f9b89868405f29d" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 42143 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 06:29:36.245584 944 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 06:29:37.885270 32523 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50278:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:29:37.885582 32523 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50278:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:29:37.890195 32523 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 15e16a76dba440648ce59c6d9f4b7a8c: Sending DeleteTablet for 3 replicas of tablet 208454ead35b4463844e580cea7ff138
I20260811 06:29:37.891064 32756 tablet_service.cc:1514] Processing DeleteTablet for tablet 208454ead35b4463844e580cea7ff138 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:29:37 UTC) from {username='kudu', principal='kudu/127.28.239.254@KRBTEST.COM'} at 127.0.0.1:34340
I20260811 06:29:37.891127 507 tablet_service.cc:1514] Processing DeleteTablet for tablet 208454ead35b4463844e580cea7ff138 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:29:37 UTC) from {username='kudu', principal='kudu/127.28.239.254@KRBTEST.COM'} at 127.0.0.1:44830
I20260811 06:29:37.891234 721 tablet_service.cc:1514] Processing DeleteTablet for tablet 208454ead35b4463844e580cea7ff138 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:29:37 UTC) from {username='kudu', principal='kudu/127.28.239.254@KRBTEST.COM'} at 127.0.0.1:35512
I20260811 06:29:37.903503 997 tablet_replica.cc:330] T 208454ead35b4463844e580cea7ff138 P c7f16b4829bd44a9952c892f26d8e8c9: stopping tablet replica
I20260811 06:29:37.903662 1000 tablet_replica.cc:330] T 208454ead35b4463844e580cea7ff138 P d184d745e54d4f629f9b89868405f29d: stopping tablet replica
I20260811 06:29:37.903885 1000 raft_consensus.cc:2229] T 208454ead35b4463844e580cea7ff138 P d184d745e54d4f629f9b89868405f29d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:29:37.903928 998 tablet_replica.cc:330] T 208454ead35b4463844e580cea7ff138 P 47f4dcc280824ca09b9c04fe9ee5c3a2: stopping tablet replica
I20260811 06:29:37.903949 997 raft_consensus.cc:2229] T 208454ead35b4463844e580cea7ff138 P c7f16b4829bd44a9952c892f26d8e8c9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:29:37.904179 1000 raft_consensus.cc:2258] T 208454ead35b4463844e580cea7ff138 P d184d745e54d4f629f9b89868405f29d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:29:37.904168 998 raft_consensus.cc:2229] T 208454ead35b4463844e580cea7ff138 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:29:37.904395 997 raft_consensus.cc:2258] T 208454ead35b4463844e580cea7ff138 P c7f16b4829bd44a9952c892f26d8e8c9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:29:37.905058 998 raft_consensus.cc:2258] T 208454ead35b4463844e580cea7ff138 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:29:37.905268 1000 ts_tablet_manager.cc:1905] T 208454ead35b4463844e580cea7ff138 P d184d745e54d4f629f9b89868405f29d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:29:37.905405 997 ts_tablet_manager.cc:1905] T 208454ead35b4463844e580cea7ff138 P c7f16b4829bd44a9952c892f26d8e8c9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:29:37.906257 998 ts_tablet_manager.cc:1905] T 208454ead35b4463844e580cea7ff138 P 47f4dcc280824ca09b9c04fe9ee5c3a2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:29:37.912290 997 ts_tablet_manager.cc:1918] T 208454ead35b4463844e580cea7ff138 P c7f16b4829bd44a9952c892f26d8e8c9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 06:29:37.912290 1000 ts_tablet_manager.cc:1918] T 208454ead35b4463844e580cea7ff138 P d184d745e54d4f629f9b89868405f29d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 06:29:37.912441 1000 log.cc:1196] T 208454ead35b4463844e580cea7ff138 P d184d745e54d4f629f9b89868405f29d: Deleting WAL directory at /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/minicluster-data/ts-2/wal/wals/208454ead35b4463844e580cea7ff138
I20260811 06:29:37.912564 997 log.cc:1196] T 208454ead35b4463844e580cea7ff138 P c7f16b4829bd44a9952c892f26d8e8c9: Deleting WAL directory at /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/minicluster-data/ts-0/wal/wals/208454ead35b4463844e580cea7ff138
I20260811 06:29:37.912959 1000 ts_tablet_manager.cc:1939] T 208454ead35b4463844e580cea7ff138 P d184d745e54d4f629f9b89868405f29d: Deleting consensus metadata
I20260811 06:29:37.913482 997 ts_tablet_manager.cc:1939] T 208454ead35b4463844e580cea7ff138 P c7f16b4829bd44a9952c892f26d8e8c9: Deleting consensus metadata
I20260811 06:29:37.914587 32497 catalog_manager.cc:4841] TS d184d745e54d4f629f9b89868405f29d (127.28.239.195:42143): tablet 208454ead35b4463844e580cea7ff138 (table test-table [id=006bbd0b31164ff5810fc760d90cc93b]) successfully deleted
I20260811 06:29:37.914865 998 ts_tablet_manager.cc:1918] T 208454ead35b4463844e580cea7ff138 P 47f4dcc280824ca09b9c04fe9ee5c3a2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 06:29:37.914934 32497 catalog_manager.cc:4841] TS c7f16b4829bd44a9952c892f26d8e8c9 (127.28.239.193:39059): tablet 208454ead35b4463844e580cea7ff138 (table test-table [id=006bbd0b31164ff5810fc760d90cc93b]) successfully deleted
I20260811 06:29:37.915014 998 log.cc:1196] T 208454ead35b4463844e580cea7ff138 P 47f4dcc280824ca09b9c04fe9ee5c3a2: Deleting WAL directory at /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/minicluster-data/ts-1/wal/wals/208454ead35b4463844e580cea7ff138
I20260811 06:29:37.915511 998 ts_tablet_manager.cc:1939] T 208454ead35b4463844e580cea7ff138 P 47f4dcc280824ca09b9c04fe9ee5c3a2: Deleting consensus metadata
I20260811 06:29:37.916524 32501 catalog_manager.cc:4841] TS 47f4dcc280824ca09b9c04fe9ee5c3a2 (127.28.239.194:33025): tablet 208454ead35b4463844e580cea7ff138 (table test-table [id=006bbd0b31164ff5810fc760d90cc93b]) successfully deleted
Aug 11 06:29:37 dist-test-slave-gkw7 krb5kdc[32451](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429777, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 06:29:37 dist-test-slave-gkw7 krb5kdc[32451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429777, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:29:37.957149 1013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:37.942143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45990 (local address 127.28.239.254:46267)
0811 06:29:37.943044 (+ 901us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:37.943051 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:37.943077 (+ 26us) server_negotiation.cc:407] Connection header received
0811 06:29:37.943147 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:37.943152 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:37.943227 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:37.943358 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:37.944811 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:37.946110 (+ 1299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:37.947500 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:37.947890 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:37.947985 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:37.952085 (+ 4100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:29:37.952090 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:29:37.952094 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:29:37.952126 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:29:37.954795 (+ 2669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:37.955463 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:37.955467 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:37.955469 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:37.955597 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:37.955924 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:37.955927 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:37.955929 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:37.956182 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:29:37.956374 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:37.956624 (+ 250us) server_negotiation.cc:299] Negotiation successful
0811 06:29:37.956790 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":763,"thread_start_us":540,"threads_started":1}
I20260811 06:29:37.960728 32523 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45990:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:29:37.961367 32523 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:29:37.968416 507 tablet_service.cc:1467] Processing CreateTablet for tablet 7f164f87d5bf47579d6cebe85ad7a463 (DEFAULT_TABLE table=test-table [id=633cd58381994a64b210bc2fbce36d68]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:29:37.968415 32756 tablet_service.cc:1467] Processing CreateTablet for tablet 7f164f87d5bf47579d6cebe85ad7a463 (DEFAULT_TABLE table=test-table [id=633cd58381994a64b210bc2fbce36d68]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:29:37.968660 721 tablet_service.cc:1467] Processing CreateTablet for tablet 7f164f87d5bf47579d6cebe85ad7a463 (DEFAULT_TABLE table=test-table [id=633cd58381994a64b210bc2fbce36d68]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:29:37.968883 32756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f164f87d5bf47579d6cebe85ad7a463. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:29:37.968904 507 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f164f87d5bf47579d6cebe85ad7a463. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:29:37.969162 721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f164f87d5bf47579d6cebe85ad7a463. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:29:37.974709 1015 tablet_bootstrap.cc:492] T 7f164f87d5bf47579d6cebe85ad7a463 P d184d745e54d4f629f9b89868405f29d: Bootstrap starting.
I20260811 06:29:37.976303 1015 tablet_bootstrap.cc:654] T 7f164f87d5bf47579d6cebe85ad7a463 P d184d745e54d4f629f9b89868405f29d: Neither blocks nor log segments found. Creating new log.
I20260811 06:29:37.977056 1017 tablet_bootstrap.cc:492] T 7f164f87d5bf47579d6cebe85ad7a463 P 47f4dcc280824ca09b9c04fe9ee5c3a2: Bootstrap starting.
I20260811 06:29:37.977465 1020 tablet_bootstrap.cc:492] T 7f164f87d5bf47579d6cebe85ad7a463 P c7f16b4829bd44a9952c892f26d8e8c9: Bootstrap starting.
I20260811 06:29:37.978793 1015 tablet_bootstrap.cc:492] T 7f164f87d5bf47579d6cebe85ad7a463 P d184d745e54d4f629f9b89868405f29d: No bootstrap required, opened a new log
I20260811 06:29:37.978961 1015 ts_tablet_manager.cc:1397] T 7f164f87d5bf47579d6cebe85ad7a463 P d184d745e54d4f629f9b89868405f29d: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20260811 06:29:37.979071 1017 tablet_bootstrap.cc:654] T 7f164f87d5bf47579d6cebe85ad7a463 P 47f4dcc280824ca09b9c04fe9ee5c3a2: Neither blocks nor log segments found. Creating new log.
I20260811 06:29:37.979427 1020 tablet_bootstrap.cc:654] T 7f164f87d5bf47579d6cebe85ad7a463 P c7f16b4829bd44a9952c892f26d8e8c9: Neither blocks nor log segments found. Creating new log.
I20260811 06:29:37.979914 1015 raft_consensus.cc:348] T 7f164f87d5bf47579d6cebe85ad7a463 P d184d745e54d4f629f9b89868405f29d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7f16b4829bd44a9952c892f26d8e8c9" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 39059 } } peers { permanent_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 33025 } } peers { permanent_uuid: "d184d745e54d4f629f9b89868405f29d" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 42143 } }
I20260811 06:29:37.980185 1015 raft_consensus.cc:374] T 7f164f87d5bf47579d6cebe85ad7a463 P d184d745e54d4f629f9b89868405f29d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:29:37.980293 1015 raft_consensus.cc:729] T 7f164f87d5bf47579d6cebe85ad7a463 P d184d745e54d4f629f9b89868405f29d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d184d745e54d4f629f9b89868405f29d, State: Initialized, Role: FOLLOWER
I20260811 06:29:37.980567 1015 consensus_queue.cc:260] T 7f164f87d5bf47579d6cebe85ad7a463 P d184d745e54d4f629f9b89868405f29d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7f16b4829bd44a9952c892f26d8e8c9" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 39059 } } peers { permanent_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 33025 } } peers { permanent_uuid: "d184d745e54d4f629f9b89868405f29d" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 42143 } }
I20260811 06:29:37.981590 1015 ts_tablet_manager.cc:1428] T 7f164f87d5bf47579d6cebe85ad7a463 P d184d745e54d4f629f9b89868405f29d: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260811 06:29:37.989202 1020 tablet_bootstrap.cc:492] T 7f164f87d5bf47579d6cebe85ad7a463 P c7f16b4829bd44a9952c892f26d8e8c9: No bootstrap required, opened a new log
I20260811 06:29:37.989223 1017 tablet_bootstrap.cc:492] T 7f164f87d5bf47579d6cebe85ad7a463 P 47f4dcc280824ca09b9c04fe9ee5c3a2: No bootstrap required, opened a new log
I20260811 06:29:37.989384 1020 ts_tablet_manager.cc:1397] T 7f164f87d5bf47579d6cebe85ad7a463 P c7f16b4829bd44a9952c892f26d8e8c9: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.007s
I20260811 06:29:37.989383 1017 ts_tablet_manager.cc:1397] T 7f164f87d5bf47579d6cebe85ad7a463 P 47f4dcc280824ca09b9c04fe9ee5c3a2: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.005s
I20260811 06:29:37.990294 1020 raft_consensus.cc:348] T 7f164f87d5bf47579d6cebe85ad7a463 P c7f16b4829bd44a9952c892f26d8e8c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7f16b4829bd44a9952c892f26d8e8c9" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 39059 } } peers { permanent_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 33025 } } peers { permanent_uuid: "d184d745e54d4f629f9b89868405f29d" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 42143 } }
I20260811 06:29:37.990273 1017 raft_consensus.cc:348] T 7f164f87d5bf47579d6cebe85ad7a463 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7f16b4829bd44a9952c892f26d8e8c9" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 39059 } } peers { permanent_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 33025 } } peers { permanent_uuid: "d184d745e54d4f629f9b89868405f29d" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 42143 } }
I20260811 06:29:37.990546 1020 raft_consensus.cc:374] T 7f164f87d5bf47579d6cebe85ad7a463 P c7f16b4829bd44a9952c892f26d8e8c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:29:37.990548 1017 raft_consensus.cc:374] T 7f164f87d5bf47579d6cebe85ad7a463 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:29:37.990658 1020 raft_consensus.cc:729] T 7f164f87d5bf47579d6cebe85ad7a463 P c7f16b4829bd44a9952c892f26d8e8c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7f16b4829bd44a9952c892f26d8e8c9, State: Initialized, Role: FOLLOWER
I20260811 06:29:37.990666 1017 raft_consensus.cc:729] T 7f164f87d5bf47579d6cebe85ad7a463 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47f4dcc280824ca09b9c04fe9ee5c3a2, State: Initialized, Role: FOLLOWER
I20260811 06:29:37.990916 1020 consensus_queue.cc:260] T 7f164f87d5bf47579d6cebe85ad7a463 P c7f16b4829bd44a9952c892f26d8e8c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7f16b4829bd44a9952c892f26d8e8c9" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 39059 } } peers { permanent_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 33025 } } peers { permanent_uuid: "d184d745e54d4f629f9b89868405f29d" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 42143 } }
I20260811 06:29:37.990916 1017 consensus_queue.cc:260] T 7f164f87d5bf47579d6cebe85ad7a463 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7f16b4829bd44a9952c892f26d8e8c9" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 39059 } } peers { permanent_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 33025 } } peers { permanent_uuid: "d184d745e54d4f629f9b89868405f29d" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 42143 } }
I20260811 06:29:37.991624 1017 ts_tablet_manager.cc:1428] T 7f164f87d5bf47579d6cebe85ad7a463 P 47f4dcc280824ca09b9c04fe9ee5c3a2: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260811 06:29:37.991969 1020 ts_tablet_manager.cc:1428] T 7f164f87d5bf47579d6cebe85ad7a463 P c7f16b4829bd44a9952c892f26d8e8c9: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 06:29:38.115772 988 raft_consensus.cc:482] T 7f164f87d5bf47579d6cebe85ad7a463 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:29:38.116024 988 raft_consensus.cc:504] T 7f164f87d5bf47579d6cebe85ad7a463 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7f16b4829bd44a9952c892f26d8e8c9" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 39059 } } peers { permanent_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 33025 } } peers { permanent_uuid: "d184d745e54d4f629f9b89868405f29d" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 42143 } }
I20260811 06:29:38.116735 988 leader_election.cc:288] T 7f164f87d5bf47579d6cebe85ad7a463 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c7f16b4829bd44a9952c892f26d8e8c9 (127.28.239.193:39059), d184d745e54d4f629f9b89868405f29d (127.28.239.195:42143)
I20260811 06:29:38.117580 760 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f164f87d5bf47579d6cebe85ad7a463" candidate_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d184d745e54d4f629f9b89868405f29d" is_pre_election: true
I20260811 06:29:38.117604 319 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f164f87d5bf47579d6cebe85ad7a463" candidate_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7f16b4829bd44a9952c892f26d8e8c9" is_pre_election: true
I20260811 06:29:38.117863 319 raft_consensus.cc:2454] T 7f164f87d5bf47579d6cebe85ad7a463 P c7f16b4829bd44a9952c892f26d8e8c9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 47f4dcc280824ca09b9c04fe9ee5c3a2 in term 0.
I20260811 06:29:38.117885 760 raft_consensus.cc:2454] T 7f164f87d5bf47579d6cebe85ad7a463 P d184d745e54d4f629f9b89868405f29d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 47f4dcc280824ca09b9c04fe9ee5c3a2 in term 0.
I20260811 06:29:38.118428 431 leader_election.cc:302] T 7f164f87d5bf47579d6cebe85ad7a463 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47f4dcc280824ca09b9c04fe9ee5c3a2, c7f16b4829bd44a9952c892f26d8e8c9; no voters:
I20260811 06:29:38.118709 988 raft_consensus.cc:2781] T 7f164f87d5bf47579d6cebe85ad7a463 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:29:38.118849 988 raft_consensus.cc:482] T 7f164f87d5bf47579d6cebe85ad7a463 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:29:38.118943 988 raft_consensus.cc:3037] T 7f164f87d5bf47579d6cebe85ad7a463 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:29:38.121047 988 raft_consensus.cc:504] T 7f164f87d5bf47579d6cebe85ad7a463 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7f16b4829bd44a9952c892f26d8e8c9" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 39059 } } peers { permanent_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 33025 } } peers { permanent_uuid: "d184d745e54d4f629f9b89868405f29d" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 42143 } }
I20260811 06:29:38.121632 988 leader_election.cc:288] T 7f164f87d5bf47579d6cebe85ad7a463 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [CANDIDATE]: Term 1 election: Requested vote from peers c7f16b4829bd44a9952c892f26d8e8c9 (127.28.239.193:39059), d184d745e54d4f629f9b89868405f29d (127.28.239.195:42143)
I20260811 06:29:38.122161 319 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f164f87d5bf47579d6cebe85ad7a463" candidate_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7f16b4829bd44a9952c892f26d8e8c9"
I20260811 06:29:38.122206 760 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f164f87d5bf47579d6cebe85ad7a463" candidate_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d184d745e54d4f629f9b89868405f29d"
I20260811 06:29:38.122335 319 raft_consensus.cc:3037] T 7f164f87d5bf47579d6cebe85ad7a463 P c7f16b4829bd44a9952c892f26d8e8c9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:29:38.122510 760 raft_consensus.cc:3037] T 7f164f87d5bf47579d6cebe85ad7a463 P d184d745e54d4f629f9b89868405f29d [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:29:38.124483 319 raft_consensus.cc:2454] T 7f164f87d5bf47579d6cebe85ad7a463 P c7f16b4829bd44a9952c892f26d8e8c9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47f4dcc280824ca09b9c04fe9ee5c3a2 in term 1.
I20260811 06:29:38.124603 760 raft_consensus.cc:2454] T 7f164f87d5bf47579d6cebe85ad7a463 P d184d745e54d4f629f9b89868405f29d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47f4dcc280824ca09b9c04fe9ee5c3a2 in term 1.
I20260811 06:29:38.124943 431 leader_election.cc:302] T 7f164f87d5bf47579d6cebe85ad7a463 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47f4dcc280824ca09b9c04fe9ee5c3a2, c7f16b4829bd44a9952c892f26d8e8c9; no voters:
I20260811 06:29:38.125265 988 raft_consensus.cc:2781] T 7f164f87d5bf47579d6cebe85ad7a463 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:29:38.125423 988 raft_consensus.cc:686] T 7f164f87d5bf47579d6cebe85ad7a463 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [term 1 LEADER]: Becoming Leader. State: Replica: 47f4dcc280824ca09b9c04fe9ee5c3a2, State: Running, Role: LEADER
I20260811 06:29:38.125741 988 consensus_queue.cc:237] T 7f164f87d5bf47579d6cebe85ad7a463 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7f16b4829bd44a9952c892f26d8e8c9" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 39059 } } peers { permanent_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 33025 } } peers { permanent_uuid: "d184d745e54d4f629f9b89868405f29d" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 42143 } }
I20260811 06:29:38.128600 32521 catalog_manager.cc:5495] T 7f164f87d5bf47579d6cebe85ad7a463 P 47f4dcc280824ca09b9c04fe9ee5c3a2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 47f4dcc280824ca09b9c04fe9ee5c3a2 (127.28.239.194). New cstate: current_term: 1 leader_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7f16b4829bd44a9952c892f26d8e8c9" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 39059 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 33025 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d184d745e54d4f629f9b89868405f29d" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 42143 } health_report { overall_health: UNKNOWN } } }
I20260811 06:29:38.188812 1035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:38.182586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53272 (local address 127.28.239.194:33025)
0811 06:29:38.183333 (+ 747us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:38.183340 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:38.183366 (+ 26us) server_negotiation.cc:407] Connection header received
0811 06:29:38.183575 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:38.183581 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:38.183665 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:38.183776 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:29:38.184222 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:38.185150 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:38.186701 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:38.187229 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:38.187381 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:38.187897 (+ 516us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:29:38.188020 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:29:38.188144 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:38.188316 (+ 172us) server_negotiation.cc:299] Negotiation successful
0811 06:29:38.188401 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":600,"thread_start_us":414,"threads_started":1}
I20260811 06:29:38.198848 760 raft_consensus.cc:1261] T 7f164f87d5bf47579d6cebe85ad7a463 P d184d745e54d4f629f9b89868405f29d [term 1 FOLLOWER]: Refusing update from remote peer 47f4dcc280824ca09b9c04fe9ee5c3a2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:29:38.198877 319 raft_consensus.cc:1261] T 7f164f87d5bf47579d6cebe85ad7a463 P c7f16b4829bd44a9952c892f26d8e8c9 [term 1 FOLLOWER]: Refusing update from remote peer 47f4dcc280824ca09b9c04fe9ee5c3a2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:29:38.199424 964 consensus_queue.cc:1035] T 7f164f87d5bf47579d6cebe85ad7a463 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7f16b4829bd44a9952c892f26d8e8c9" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 39059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:29:38.199740 988 consensus_queue.cc:1035] T 7f164f87d5bf47579d6cebe85ad7a463 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d184d745e54d4f629f9b89868405f29d" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 42143 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:29:38.248956 32523 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45990:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:29:38.249171 32523 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45990:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:29:38.252173 32523 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 15e16a76dba440648ce59c6d9f4b7a8c: Sending DeleteTablet for 3 replicas of tablet 7f164f87d5bf47579d6cebe85ad7a463
I20260811 06:29:38.253031 32756 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f164f87d5bf47579d6cebe85ad7a463 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:29:38 UTC) from {username='kudu', principal='kudu/127.28.239.254@KRBTEST.COM'} at 127.0.0.1:34340
I20260811 06:29:38.253093 506 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f164f87d5bf47579d6cebe85ad7a463 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:29:38 UTC) from {username='kudu', principal='kudu/127.28.239.254@KRBTEST.COM'} at 127.0.0.1:44830
I20260811 06:29:38.253407 998 tablet_replica.cc:330] T 7f164f87d5bf47579d6cebe85ad7a463 P 47f4dcc280824ca09b9c04fe9ee5c3a2: stopping tablet replica
I20260811 06:29:38.253401 721 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f164f87d5bf47579d6cebe85ad7a463 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:29:38 UTC) from {username='kudu', principal='kudu/127.28.239.254@KRBTEST.COM'} at 127.0.0.1:35512
I20260811 06:29:38.253659 998 raft_consensus.cc:2229] T 7f164f87d5bf47579d6cebe85ad7a463 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:29:38.253707 1000 tablet_replica.cc:330] T 7f164f87d5bf47579d6cebe85ad7a463 P d184d745e54d4f629f9b89868405f29d: stopping tablet replica
I20260811 06:29:38.253901 1000 raft_consensus.cc:2229] T 7f164f87d5bf47579d6cebe85ad7a463 P d184d745e54d4f629f9b89868405f29d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:29:38.254037 998 raft_consensus.cc:2258] T 7f164f87d5bf47579d6cebe85ad7a463 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:29:38.254094 1000 raft_consensus.cc:2258] T 7f164f87d5bf47579d6cebe85ad7a463 P d184d745e54d4f629f9b89868405f29d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:29:38.254341 997 tablet_replica.cc:330] T 7f164f87d5bf47579d6cebe85ad7a463 P c7f16b4829bd44a9952c892f26d8e8c9: stopping tablet replica
I20260811 06:29:38.254303 32523 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45990:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 06:29:38.254565 997 raft_consensus.cc:2229] T 7f164f87d5bf47579d6cebe85ad7a463 P c7f16b4829bd44a9952c892f26d8e8c9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:29:38.254784 997 raft_consensus.cc:2258] T 7f164f87d5bf47579d6cebe85ad7a463 P c7f16b4829bd44a9952c892f26d8e8c9 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260811 06:29:38.255038 32523 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:29:38.255331 998 ts_tablet_manager.cc:1905] T 7f164f87d5bf47579d6cebe85ad7a463 P 47f4dcc280824ca09b9c04fe9ee5c3a2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:29:38.255823 997 ts_tablet_manager.cc:1905] T 7f164f87d5bf47579d6cebe85ad7a463 P c7f16b4829bd44a9952c892f26d8e8c9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:29:38.261500 32756 tablet_service.cc:1467] Processing CreateTablet for tablet 92d144aa06344b998fd8b6fa0a07bc11 (DEFAULT_TABLE table=test-table [id=4590625502564280879402ab940b6d34]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:29:38.262029 32756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92d144aa06344b998fd8b6fa0a07bc11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:29:38.262040 721 tablet_service.cc:1467] Processing CreateTablet for tablet 92d144aa06344b998fd8b6fa0a07bc11 (DEFAULT_TABLE table=test-table [id=4590625502564280879402ab940b6d34]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:29:38.262441 997 ts_tablet_manager.cc:1918] T 7f164f87d5bf47579d6cebe85ad7a463 P c7f16b4829bd44a9952c892f26d8e8c9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 06:29:38.262501 721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92d144aa06344b998fd8b6fa0a07bc11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:29:38.262593 997 log.cc:1196] T 7f164f87d5bf47579d6cebe85ad7a463 P c7f16b4829bd44a9952c892f26d8e8c9: Deleting WAL directory at /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/minicluster-data/ts-0/wal/wals/7f164f87d5bf47579d6cebe85ad7a463
I20260811 06:29:38.263135 997 ts_tablet_manager.cc:1939] T 7f164f87d5bf47579d6cebe85ad7a463 P c7f16b4829bd44a9952c892f26d8e8c9: Deleting consensus metadata
I20260811 06:29:38.264257 32497 catalog_manager.cc:4841] TS c7f16b4829bd44a9952c892f26d8e8c9 (127.28.239.193:39059): tablet 7f164f87d5bf47579d6cebe85ad7a463 (table test-table [id=633cd58381994a64b210bc2fbce36d68]) successfully deleted
I20260811 06:29:38.264572 506 tablet_service.cc:1467] Processing CreateTablet for tablet 92d144aa06344b998fd8b6fa0a07bc11 (DEFAULT_TABLE table=test-table [id=4590625502564280879402ab940b6d34]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:29:38.264966 506 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92d144aa06344b998fd8b6fa0a07bc11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:29:38.266826 1020 tablet_bootstrap.cc:492] T 92d144aa06344b998fd8b6fa0a07bc11 P c7f16b4829bd44a9952c892f26d8e8c9: Bootstrap starting.
I20260811 06:29:38.268170 1020 tablet_bootstrap.cc:654] T 92d144aa06344b998fd8b6fa0a07bc11 P c7f16b4829bd44a9952c892f26d8e8c9: Neither blocks nor log segments found. Creating new log.
I20260811 06:29:38.268847 1015 tablet_bootstrap.cc:492] T 92d144aa06344b998fd8b6fa0a07bc11 P d184d745e54d4f629f9b89868405f29d: Bootstrap starting.
I20260811 06:29:38.269546 998 ts_tablet_manager.cc:1918] T 7f164f87d5bf47579d6cebe85ad7a463 P 47f4dcc280824ca09b9c04fe9ee5c3a2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 06:29:38.269701 998 log.cc:1196] T 7f164f87d5bf47579d6cebe85ad7a463 P 47f4dcc280824ca09b9c04fe9ee5c3a2: Deleting WAL directory at /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/minicluster-data/ts-1/wal/wals/7f164f87d5bf47579d6cebe85ad7a463
I20260811 06:29:38.270179 998 ts_tablet_manager.cc:1939] T 7f164f87d5bf47579d6cebe85ad7a463 P 47f4dcc280824ca09b9c04fe9ee5c3a2: Deleting consensus metadata
I20260811 06:29:38.271000 1015 tablet_bootstrap.cc:654] T 92d144aa06344b998fd8b6fa0a07bc11 P d184d745e54d4f629f9b89868405f29d: Neither blocks nor log segments found. Creating new log.
I20260811 06:29:38.271123 1017 tablet_bootstrap.cc:492] T 92d144aa06344b998fd8b6fa0a07bc11 P 47f4dcc280824ca09b9c04fe9ee5c3a2: Bootstrap starting.
I20260811 06:29:38.271759 1000 ts_tablet_manager.cc:1905] T 7f164f87d5bf47579d6cebe85ad7a463 P d184d745e54d4f629f9b89868405f29d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:29:38.272845 1017 tablet_bootstrap.cc:654] T 92d144aa06344b998fd8b6fa0a07bc11 P 47f4dcc280824ca09b9c04fe9ee5c3a2: Neither blocks nor log segments found. Creating new log.
I20260811 06:29:38.273758 32501 catalog_manager.cc:4841] TS 47f4dcc280824ca09b9c04fe9ee5c3a2 (127.28.239.194:33025): tablet 7f164f87d5bf47579d6cebe85ad7a463 (table test-table [id=633cd58381994a64b210bc2fbce36d68]) successfully deleted
I20260811 06:29:38.273845 1015 tablet_bootstrap.cc:492] T 92d144aa06344b998fd8b6fa0a07bc11 P d184d745e54d4f629f9b89868405f29d: No bootstrap required, opened a new log
I20260811 06:29:38.274008 1015 ts_tablet_manager.cc:1397] T 92d144aa06344b998fd8b6fa0a07bc11 P d184d745e54d4f629f9b89868405f29d: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.002s
I20260811 06:29:38.274860 1015 raft_consensus.cc:348] T 92d144aa06344b998fd8b6fa0a07bc11 P d184d745e54d4f629f9b89868405f29d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7f16b4829bd44a9952c892f26d8e8c9" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 39059 } } peers { permanent_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 33025 } } peers { permanent_uuid: "d184d745e54d4f629f9b89868405f29d" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 42143 } }
I20260811 06:29:38.275156 1015 raft_consensus.cc:374] T 92d144aa06344b998fd8b6fa0a07bc11 P d184d745e54d4f629f9b89868405f29d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:29:38.275255 1015 raft_consensus.cc:729] T 92d144aa06344b998fd8b6fa0a07bc11 P d184d745e54d4f629f9b89868405f29d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d184d745e54d4f629f9b89868405f29d, State: Initialized, Role: FOLLOWER
I20260811 06:29:38.275584 1020 tablet_bootstrap.cc:492] T 92d144aa06344b998fd8b6fa0a07bc11 P c7f16b4829bd44a9952c892f26d8e8c9: No bootstrap required, opened a new log
I20260811 06:29:38.275513 1015 consensus_queue.cc:260] T 92d144aa06344b998fd8b6fa0a07bc11 P d184d745e54d4f629f9b89868405f29d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7f16b4829bd44a9952c892f26d8e8c9" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 39059 } } peers { permanent_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 33025 } } peers { permanent_uuid: "d184d745e54d4f629f9b89868405f29d" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 42143 } }
I20260811 06:29:38.275895 1020 ts_tablet_manager.cc:1397] T 92d144aa06344b998fd8b6fa0a07bc11 P c7f16b4829bd44a9952c892f26d8e8c9: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.004s
I20260811 06:29:38.276172 1015 ts_tablet_manager.cc:1428] T 92d144aa06344b998fd8b6fa0a07bc11 P d184d745e54d4f629f9b89868405f29d: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260811 06:29:38.276724 1020 raft_consensus.cc:348] T 92d144aa06344b998fd8b6fa0a07bc11 P c7f16b4829bd44a9952c892f26d8e8c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7f16b4829bd44a9952c892f26d8e8c9" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 39059 } } peers { permanent_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 33025 } } peers { permanent_uuid: "d184d745e54d4f629f9b89868405f29d" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 42143 } }
I20260811 06:29:38.276965 1020 raft_consensus.cc:374] T 92d144aa06344b998fd8b6fa0a07bc11 P c7f16b4829bd44a9952c892f26d8e8c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:29:38.277050 1020 raft_consensus.cc:729] T 92d144aa06344b998fd8b6fa0a07bc11 P c7f16b4829bd44a9952c892f26d8e8c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7f16b4829bd44a9952c892f26d8e8c9, State: Initialized, Role: FOLLOWER
I20260811 06:29:38.277284 1020 consensus_queue.cc:260] T 92d144aa06344b998fd8b6fa0a07bc11 P c7f16b4829bd44a9952c892f26d8e8c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7f16b4829bd44a9952c892f26d8e8c9" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 39059 } } peers { permanent_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 33025 } } peers { permanent_uuid: "d184d745e54d4f629f9b89868405f29d" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 42143 } }
I20260811 06:29:38.277954 1020 ts_tablet_manager.cc:1428] T 92d144aa06344b998fd8b6fa0a07bc11 P c7f16b4829bd44a9952c892f26d8e8c9: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 06:29:38.278817 1000 ts_tablet_manager.cc:1918] T 7f164f87d5bf47579d6cebe85ad7a463 P d184d745e54d4f629f9b89868405f29d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 06:29:38.278972 1000 log.cc:1196] T 7f164f87d5bf47579d6cebe85ad7a463 P d184d745e54d4f629f9b89868405f29d: Deleting WAL directory at /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/minicluster-data/ts-2/wal/wals/7f164f87d5bf47579d6cebe85ad7a463
I20260811 06:29:38.279515 1017 tablet_bootstrap.cc:492] T 92d144aa06344b998fd8b6fa0a07bc11 P 47f4dcc280824ca09b9c04fe9ee5c3a2: No bootstrap required, opened a new log
I20260811 06:29:38.279505 1000 ts_tablet_manager.cc:1939] T 7f164f87d5bf47579d6cebe85ad7a463 P d184d745e54d4f629f9b89868405f29d: Deleting consensus metadata
I20260811 06:29:38.279671 1017 ts_tablet_manager.cc:1397] T 92d144aa06344b998fd8b6fa0a07bc11 P 47f4dcc280824ca09b9c04fe9ee5c3a2: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.001s
I20260811 06:29:38.280511 32497 catalog_manager.cc:4841] TS d184d745e54d4f629f9b89868405f29d (127.28.239.195:42143): tablet 7f164f87d5bf47579d6cebe85ad7a463 (table test-table [id=633cd58381994a64b210bc2fbce36d68]) successfully deleted
I20260811 06:29:38.280452 1017 raft_consensus.cc:348] T 92d144aa06344b998fd8b6fa0a07bc11 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7f16b4829bd44a9952c892f26d8e8c9" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 39059 } } peers { permanent_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 33025 } } peers { permanent_uuid: "d184d745e54d4f629f9b89868405f29d" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 42143 } }
I20260811 06:29:38.280722 1017 raft_consensus.cc:374] T 92d144aa06344b998fd8b6fa0a07bc11 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:29:38.280830 1017 raft_consensus.cc:729] T 92d144aa06344b998fd8b6fa0a07bc11 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47f4dcc280824ca09b9c04fe9ee5c3a2, State: Initialized, Role: FOLLOWER
I20260811 06:29:38.281081 1017 consensus_queue.cc:260] T 92d144aa06344b998fd8b6fa0a07bc11 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7f16b4829bd44a9952c892f26d8e8c9" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 39059 } } peers { permanent_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 33025 } } peers { permanent_uuid: "d184d745e54d4f629f9b89868405f29d" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 42143 } }
I20260811 06:29:38.281751 1017 ts_tablet_manager.cc:1428] T 92d144aa06344b998fd8b6fa0a07bc11 P 47f4dcc280824ca09b9c04fe9ee5c3a2: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 06:29:38.488950 988 raft_consensus.cc:482] T 92d144aa06344b998fd8b6fa0a07bc11 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:29:38.489224 988 raft_consensus.cc:504] T 92d144aa06344b998fd8b6fa0a07bc11 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7f16b4829bd44a9952c892f26d8e8c9" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 39059 } } peers { permanent_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 33025 } } peers { permanent_uuid: "d184d745e54d4f629f9b89868405f29d" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 42143 } }
I20260811 06:29:38.489995 988 leader_election.cc:288] T 92d144aa06344b998fd8b6fa0a07bc11 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c7f16b4829bd44a9952c892f26d8e8c9 (127.28.239.193:39059), d184d745e54d4f629f9b89868405f29d (127.28.239.195:42143)
I20260811 06:29:38.490713 760 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92d144aa06344b998fd8b6fa0a07bc11" candidate_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d184d745e54d4f629f9b89868405f29d" is_pre_election: true
I20260811 06:29:38.490712 321 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92d144aa06344b998fd8b6fa0a07bc11" candidate_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7f16b4829bd44a9952c892f26d8e8c9" is_pre_election: true
I20260811 06:29:38.491022 760 raft_consensus.cc:2454] T 92d144aa06344b998fd8b6fa0a07bc11 P d184d745e54d4f629f9b89868405f29d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 47f4dcc280824ca09b9c04fe9ee5c3a2 in term 0.
I20260811 06:29:38.491022 321 raft_consensus.cc:2454] T 92d144aa06344b998fd8b6fa0a07bc11 P c7f16b4829bd44a9952c892f26d8e8c9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 47f4dcc280824ca09b9c04fe9ee5c3a2 in term 0.
I20260811 06:29:38.491485 431 leader_election.cc:302] T 92d144aa06344b998fd8b6fa0a07bc11 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47f4dcc280824ca09b9c04fe9ee5c3a2, c7f16b4829bd44a9952c892f26d8e8c9; no voters:
I20260811 06:29:38.491776 988 raft_consensus.cc:2781] T 92d144aa06344b998fd8b6fa0a07bc11 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:29:38.491904 988 raft_consensus.cc:482] T 92d144aa06344b998fd8b6fa0a07bc11 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:29:38.492003 988 raft_consensus.cc:3037] T 92d144aa06344b998fd8b6fa0a07bc11 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:29:38.494171 988 raft_consensus.cc:504] T 92d144aa06344b998fd8b6fa0a07bc11 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7f16b4829bd44a9952c892f26d8e8c9" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 39059 } } peers { permanent_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 33025 } } peers { permanent_uuid: "d184d745e54d4f629f9b89868405f29d" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 42143 } }
I20260811 06:29:38.494720 988 leader_election.cc:288] T 92d144aa06344b998fd8b6fa0a07bc11 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [CANDIDATE]: Term 1 election: Requested vote from peers c7f16b4829bd44a9952c892f26d8e8c9 (127.28.239.193:39059), d184d745e54d4f629f9b89868405f29d (127.28.239.195:42143)
I20260811 06:29:38.495194 321 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92d144aa06344b998fd8b6fa0a07bc11" candidate_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7f16b4829bd44a9952c892f26d8e8c9"
I20260811 06:29:38.495301 760 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92d144aa06344b998fd8b6fa0a07bc11" candidate_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d184d745e54d4f629f9b89868405f29d"
I20260811 06:29:38.495468 321 raft_consensus.cc:3037] T 92d144aa06344b998fd8b6fa0a07bc11 P c7f16b4829bd44a9952c892f26d8e8c9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:29:38.495573 760 raft_consensus.cc:3037] T 92d144aa06344b998fd8b6fa0a07bc11 P d184d745e54d4f629f9b89868405f29d [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:29:38.497615 321 raft_consensus.cc:2454] T 92d144aa06344b998fd8b6fa0a07bc11 P c7f16b4829bd44a9952c892f26d8e8c9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47f4dcc280824ca09b9c04fe9ee5c3a2 in term 1.
I20260811 06:29:38.497725 760 raft_consensus.cc:2454] T 92d144aa06344b998fd8b6fa0a07bc11 P d184d745e54d4f629f9b89868405f29d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47f4dcc280824ca09b9c04fe9ee5c3a2 in term 1.
I20260811 06:29:38.498055 431 leader_election.cc:302] T 92d144aa06344b998fd8b6fa0a07bc11 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47f4dcc280824ca09b9c04fe9ee5c3a2, c7f16b4829bd44a9952c892f26d8e8c9; no voters:
I20260811 06:29:38.498449 988 raft_consensus.cc:2781] T 92d144aa06344b998fd8b6fa0a07bc11 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:29:38.498603 988 raft_consensus.cc:686] T 92d144aa06344b998fd8b6fa0a07bc11 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [term 1 LEADER]: Becoming Leader. State: Replica: 47f4dcc280824ca09b9c04fe9ee5c3a2, State: Running, Role: LEADER
I20260811 06:29:38.498893 988 consensus_queue.cc:237] T 92d144aa06344b998fd8b6fa0a07bc11 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7f16b4829bd44a9952c892f26d8e8c9" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 39059 } } peers { permanent_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 33025 } } peers { permanent_uuid: "d184d745e54d4f629f9b89868405f29d" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 42143 } }
I20260811 06:29:38.501355 32523 catalog_manager.cc:5495] T 92d144aa06344b998fd8b6fa0a07bc11 P 47f4dcc280824ca09b9c04fe9ee5c3a2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 47f4dcc280824ca09b9c04fe9ee5c3a2 (127.28.239.194). New cstate: current_term: 1 leader_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7f16b4829bd44a9952c892f26d8e8c9" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 39059 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "47f4dcc280824ca09b9c04fe9ee5c3a2" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 33025 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d184d745e54d4f629f9b89868405f29d" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 42143 } health_report { overall_health: UNKNOWN } } }
I20260811 06:29:38.535938 760 raft_consensus.cc:1261] T 92d144aa06344b998fd8b6fa0a07bc11 P d184d745e54d4f629f9b89868405f29d [term 1 FOLLOWER]: Refusing update from remote peer 47f4dcc280824ca09b9c04fe9ee5c3a2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:29:38.536084 321 raft_consensus.cc:1261] T 92d144aa06344b998fd8b6fa0a07bc11 P c7f16b4829bd44a9952c892f26d8e8c9 [term 1 FOLLOWER]: Refusing update from remote peer 47f4dcc280824ca09b9c04fe9ee5c3a2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:29:38.536520 988 consensus_queue.cc:1035] T 92d144aa06344b998fd8b6fa0a07bc11 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d184d745e54d4f629f9b89868405f29d" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 42143 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:29:38.536834 967 consensus_queue.cc:1035] T 92d144aa06344b998fd8b6fa0a07bc11 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7f16b4829bd44a9952c892f26d8e8c9" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 39059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:29:38.547169 969 mvcc.cc:205] Tried to move back new op lower bound from 7317216372878413824 to 7317216372733304832. Current Snapshot: MvccSnapshot[applied={T|T < 7317216372878413824}]
W20260811 06:29:38.556424 965 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 06:29:40.147203 32523 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45990:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:29:40.147420 32523 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45990:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:29:40.151448 32523 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 15e16a76dba440648ce59c6d9f4b7a8c: Sending DeleteTablet for 3 replicas of tablet 92d144aa06344b998fd8b6fa0a07bc11
I20260811 06:29:40.152261 32756 tablet_service.cc:1514] Processing DeleteTablet for tablet 92d144aa06344b998fd8b6fa0a07bc11 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:29:40 UTC) from {username='kudu', principal='kudu/127.28.239.254@KRBTEST.COM'} at 127.0.0.1:34340
I20260811 06:29:40.152319 506 tablet_service.cc:1514] Processing DeleteTablet for tablet 92d144aa06344b998fd8b6fa0a07bc11 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:29:40 UTC) from {username='kudu', principal='kudu/127.28.239.254@KRBTEST.COM'} at 127.0.0.1:44830
I20260811 06:29:40.152387 721 tablet_service.cc:1514] Processing DeleteTablet for tablet 92d144aa06344b998fd8b6fa0a07bc11 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:29:40 UTC) from {username='kudu', principal='kudu/127.28.239.254@KRBTEST.COM'} at 127.0.0.1:35512
I20260811 06:29:40.153326 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 32645
I20260811 06:29:40.154583 1090 tablet_replica.cc:330] T 92d144aa06344b998fd8b6fa0a07bc11 P d184d745e54d4f629f9b89868405f29d: stopping tablet replica
I20260811 06:29:40.154896 1090 raft_consensus.cc:2229] T 92d144aa06344b998fd8b6fa0a07bc11 P d184d745e54d4f629f9b89868405f29d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:29:40.155274 1090 raft_consensus.cc:2258] T 92d144aa06344b998fd8b6fa0a07bc11 P d184d745e54d4f629f9b89868405f29d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:29:40.155889 1088 tablet_replica.cc:330] T 92d144aa06344b998fd8b6fa0a07bc11 P 47f4dcc280824ca09b9c04fe9ee5c3a2: stopping tablet replica
I20260811 06:29:40.156102 1088 raft_consensus.cc:2229] T 92d144aa06344b998fd8b6fa0a07bc11 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:29:40.156445 1088 raft_consensus.cc:2258] T 92d144aa06344b998fd8b6fa0a07bc11 P 47f4dcc280824ca09b9c04fe9ee5c3a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:29:40.157101 1087 tablet_replica.cc:330] T 92d144aa06344b998fd8b6fa0a07bc11 P c7f16b4829bd44a9952c892f26d8e8c9: stopping tablet replica
I20260811 06:29:40.157312 1087 raft_consensus.cc:2229] T 92d144aa06344b998fd8b6fa0a07bc11 P c7f16b4829bd44a9952c892f26d8e8c9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:29:40.157301 1090 ts_tablet_manager.cc:1905] T 92d144aa06344b998fd8b6fa0a07bc11 P d184d745e54d4f629f9b89868405f29d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:29:40.157598 1087 raft_consensus.cc:2258] T 92d144aa06344b998fd8b6fa0a07bc11 P c7f16b4829bd44a9952c892f26d8e8c9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:29:40.158466 1088 ts_tablet_manager.cc:1905] T 92d144aa06344b998fd8b6fa0a07bc11 P 47f4dcc280824ca09b9c04fe9ee5c3a2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:29:40.159055 1087 ts_tablet_manager.cc:1905] T 92d144aa06344b998fd8b6fa0a07bc11 P c7f16b4829bd44a9952c892f26d8e8c9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:29:40.165707 1088 ts_tablet_manager.cc:1918] T 92d144aa06344b998fd8b6fa0a07bc11 P 47f4dcc280824ca09b9c04fe9ee5c3a2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 06:29:40.165928 1088 log.cc:1196] T 92d144aa06344b998fd8b6fa0a07bc11 P 47f4dcc280824ca09b9c04fe9ee5c3a2: Deleting WAL directory at /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/minicluster-data/ts-1/wal/wals/92d144aa06344b998fd8b6fa0a07bc11
I20260811 06:29:40.165954 1087 ts_tablet_manager.cc:1918] T 92d144aa06344b998fd8b6fa0a07bc11 P c7f16b4829bd44a9952c892f26d8e8c9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 06:29:40.166069 1087 log.cc:1196] T 92d144aa06344b998fd8b6fa0a07bc11 P c7f16b4829bd44a9952c892f26d8e8c9: Deleting WAL directory at /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/minicluster-data/ts-0/wal/wals/92d144aa06344b998fd8b6fa0a07bc11
Aug 11 06:29:40 dist-test-slave-gkw7 krb5kdc[32451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429777, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.28.239.193@KRBTEST.COM
I20260811 06:29:40.166587 1088 ts_tablet_manager.cc:1939] T 92d144aa06344b998fd8b6fa0a07bc11 P 47f4dcc280824ca09b9c04fe9ee5c3a2: Deleting consensus metadata
I20260811 06:29:40.167024 1087 ts_tablet_manager.cc:1939] T 92d144aa06344b998fd8b6fa0a07bc11 P c7f16b4829bd44a9952c892f26d8e8c9: Deleting consensus metadata
I20260811 06:29:40.167814 32501 catalog_manager.cc:4841] TS 47f4dcc280824ca09b9c04fe9ee5c3a2 (127.28.239.194:33025): tablet 92d144aa06344b998fd8b6fa0a07bc11 (table test-table [id=4590625502564280879402ab940b6d34]) successfully deleted
I20260811 06:29:40.168385 32497 catalog_manager.cc:4841] TS c7f16b4829bd44a9952c892f26d8e8c9 (127.28.239.193:39059): tablet 92d144aa06344b998fd8b6fa0a07bc11 (table test-table [id=4590625502564280879402ab940b6d34]) successfully deleted
I20260811 06:29:40.169869 1090 ts_tablet_manager.cc:1918] T 92d144aa06344b998fd8b6fa0a07bc11 P d184d745e54d4f629f9b89868405f29d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 06:29:40.170027 1090 log.cc:1196] T 92d144aa06344b998fd8b6fa0a07bc11 P d184d745e54d4f629f9b89868405f29d: Deleting WAL directory at /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786429755927508-29631-0/minicluster-data/ts-2/wal/wals/92d144aa06344b998fd8b6fa0a07bc11
I20260811 06:29:40.170451 1090 ts_tablet_manager.cc:1939] T 92d144aa06344b998fd8b6fa0a07bc11 P d184d745e54d4f629f9b89868405f29d: Deleting consensus metadata
I20260811 06:29:40.171382 32497 catalog_manager.cc:4841] TS d184d745e54d4f629f9b89868405f29d (127.28.239.195:42143): tablet 92d144aa06344b998fd8b6fa0a07bc11 (table test-table [id=4590625502564280879402ab940b6d34]) successfully deleted
I20260811 06:29:40.174479 1093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:40.154876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49202 (local address 127.28.239.193:39059)
0811 06:29:40.156146 (+ 1270us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:40.156155 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:40.156185 (+ 30us) server_negotiation.cc:407] Connection header received
0811 06:29:40.156259 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:40.156267 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:40.156377 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:40.156527 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:40.158214 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:40.159393 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:40.161242 (+ 1849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:40.161960 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:40.162079 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:40.167451 (+ 5372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:29:40.167470 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:29:40.167491 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:29:40.167531 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:29:40.171491 (+ 3960us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:40.172346 (+ 855us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:40.172357 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:40.172361 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:40.172507 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:40.172857 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:40.172862 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:40.172866 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:40.173338 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:29:40.173674 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:40.173882 (+ 208us) server_negotiation.cc:299] Negotiation successful
0811 06:29:40.174110 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1094,"thread_start_us":954,"threads_started":1}
W20260811 06:29:40.175251 389 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49202
W20260811 06:29:40.175925 29631 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 32645: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 06:29:40.176115 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 32645
I20260811 06:29:40.203379 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 407
Aug 11 06:29:40 dist-test-slave-gkw7 krb5kdc[32451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429777, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.28.239.194@KRBTEST.COM
I20260811 06:29:40.218804 1094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:40.204113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53282 (local address 127.28.239.194:33025)
0811 06:29:40.204675 (+ 562us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:40.204680 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:40.204697 (+ 17us) server_negotiation.cc:407] Connection header received
0811 06:29:40.204753 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:40.204757 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:40.204822 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:40.204918 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:40.206319 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:40.207061 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:40.208296 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:40.208692 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:40.208784 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:40.212923 (+ 4139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:29:40.212939 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:29:40.212956 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:29:40.212993 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:29:40.215914 (+ 2921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:40.216587 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:40.216595 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:40.216597 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:40.216733 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:40.217113 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:40.217118 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:40.217121 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:40.217501 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:29:40.217881 (+ 380us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:40.218207 (+ 326us) server_negotiation.cc:299] Negotiation successful
0811 06:29:40.218450 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":443,"thread_start_us":282,"threads_started":1}
W20260811 06:29:40.219476 609 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53282
W20260811 06:29:40.220088 29631 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 407: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 06:29:40.220219 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 407
I20260811 06:29:40.259524 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 622
Aug 11 06:29:40 dist-test-slave-gkw7 krb5kdc[32451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429777, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.28.239.195@KRBTEST.COM
I20260811 06:29:40.276063 1095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:40.260373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48526 (local address 127.28.239.195:42143)
0811 06:29:40.260914 (+ 541us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:40.260918 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:40.260933 (+ 15us) server_negotiation.cc:407] Connection header received
0811 06:29:40.260985 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:40.260989 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:40.261046 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:40.261178 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:40.262739 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:40.263522 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:40.264788 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:40.265204 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:40.265338 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:40.269967 (+ 4629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:29:40.269981 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:29:40.269998 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:29:40.270036 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:29:40.273119 (+ 3083us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:40.273899 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:40.273914 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:40.273917 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:40.274061 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:40.274509 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:40.274513 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:40.274515 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:40.274857 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:29:40.275189 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:40.275486 (+ 297us) server_negotiation.cc:299] Negotiation successful
0811 06:29:40.275716 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":240,"threads_started":1}
W20260811 06:29:40.276724 817 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48526
W20260811 06:29:40.277309 29631 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 622: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 06:29:40.277456 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 622
I20260811 06:29:40.303524 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 32467
I20260811 06:29:40.304759 32596 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:29:40.425176 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 32467
2026-08-11T06:29:40Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (9517 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:29:40 dist-test-slave-gkw7 krb5kdc[1103](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:29:40 dist-test-slave-gkw7 krb5kdc[1103](info): set up 2 sockets
Aug 11 06:29:40 dist-test-slave-gkw7 krb5kdc[1103](info): commencing operation
krb5kdc: starting...
W20260811 06:29:41.755352 29631 mini_kdc.cc:121] Time spent starting KDC: real 1.273s user 0.001s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
W20260811 06:29:41.911540 29631 mini_kdc.cc:253] Time spent creating user principal test-admin: real 0.156s user 0.001s sys 0.009s
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
W20260811 06:29:42.036751 29631 mini_kdc.cc:253] Time spent creating user principal test-user: real 0.125s user 0.000s sys 0.007s
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
W20260811 06:29:42.203822 29631 mini_kdc.cc:253] Time spent creating user principal joe-interloper: real 0.167s user 0.000s sys 0.007s
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/krb5kdc/test-user.keytab.
W20260811 06:29:42.599749 29631 mini_kdc.cc:302] Time spent creating keytab for test-user: real 0.396s user 0.000s sys 0.007s
Aug 11 06:29:42 dist-test-slave-gkw7 krb5kdc[1103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429782, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:29:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:29:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.239.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.239.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.239.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab.
Entry for principal kudu/127.28.239.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab.
I20260811 06:29:42.714071 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:33329
--webserver_interface=127.28.239.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab
--principal=kudu/127.28.239.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:43837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.239.254:33329
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:29:43.064859 1127 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:29:43.065199 1127 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:29:43.065294 1127 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:29:43.072777 1127 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:29:43.072898 1127 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:29:43.072947 1127 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:29:43.073005 1127 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:29:43.073073 1127 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:29:43.083173 1127 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:43837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.239.254:33329
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab
--principal=kudu/127.28.239.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:33329
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.239.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:29:43.084529 1127 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:29:43.085873 1127 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:29:43.099334 1143 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:29:43.099377 1141 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:29:43.099406 1140 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:29:43.099764 1127 server_base.cc:1034] running on GCE node
I20260811 06:29:43.100955 1127 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:29:43.102607 1127 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:29:43.103924 1127 hybrid_clock.cc:648] HybridClock initialized: now 1786429783103864 us; error 43 us; skew 500 ppm
I20260811 06:29:43.104334 1127 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:29:43 dist-test-slave-gkw7 krb5kdc[1103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429783, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:29:43.107818 1127 init.cc:377] Logged in from keytab as kudu/127.28.239.254@KRBTEST.COM (short username kudu)
I20260811 06:29:43.109494 1127 webserver.cc:460] Webserver started at http://127.28.239.254:40291/ using document root <none> and password file <none>
I20260811 06:29:43.110297 1127 fs_manager.cc:359] Metadata directory not provided
I20260811 06:29:43.110453 1127 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:29:43.110800 1127 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:29:43.114005 1127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/minicluster-data/master-0/data/instance:
uuid: "b4f0cd7764634d19ae4ec8c39eb5df30"
format_stamp: "Formatted at 2026-08-11 06:29:43 on dist-test-slave-gkw7"
server_key: "feb1dc478abfd1862d4f100002e56562"
server_key_iv: "fbccb331fa7c03d87ab3c9f288159273"
server_key_version: "encryptionkey@0"
I20260811 06:29:43.114825 1127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/minicluster-data/master-0/wal/instance:
uuid: "b4f0cd7764634d19ae4ec8c39eb5df30"
format_stamp: "Formatted at 2026-08-11 06:29:43 on dist-test-slave-gkw7"
server_key: "feb1dc478abfd1862d4f100002e56562"
server_key_iv: "fbccb331fa7c03d87ab3c9f288159273"
server_key_version: "encryptionkey@0"
I20260811 06:29:43.120370 1127 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 06:29:43.124203 1158 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:29:43.125778 1127 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 06:29:43.125975 1127 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/minicluster-data/master-0/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/minicluster-data/master-0/wal
uuid: "b4f0cd7764634d19ae4ec8c39eb5df30"
format_stamp: "Formatted at 2026-08-11 06:29:43 on dist-test-slave-gkw7"
server_key: "feb1dc478abfd1862d4f100002e56562"
server_key_iv: "fbccb331fa7c03d87ab3c9f288159273"
server_key_version: "encryptionkey@0"
I20260811 06:29:43.126188 1127 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:29:43.156279 1127 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:29:43.160626 1127 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:29:43.160948 1127 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:29:43.180768 1127 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.254:33329
I20260811 06:29:43.180792 1242 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.254:33329 every 8 connection(s)
I20260811 06:29:43.182359 1127 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
I20260811 06:29:43.186352 1243 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:29:43.189705 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 1127
I20260811 06:29:43.189968 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/minicluster-data/master-0/wal/instance
I20260811 06:29:43.190630 29631 external_mini_cluster.cc:1428] Setting key d49bf66da095fbac07653a2a28cf4f48
I20260811 06:29:43.200232 1243 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4f0cd7764634d19ae4ec8c39eb5df30: Bootstrap starting.
I20260811 06:29:43.204182 1243 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b4f0cd7764634d19ae4ec8c39eb5df30: Neither blocks nor log segments found. Creating new log.
I20260811 06:29:43.205366 1243 log.cc:824] T 00000000000000000000000000000000 P b4f0cd7764634d19ae4ec8c39eb5df30: Log is configured to *not* fsync() on all Append() calls
Aug 11 06:29:43 dist-test-slave-gkw7 krb5kdc[1103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429782, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:29:43.208716 1243 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4f0cd7764634d19ae4ec8c39eb5df30: No bootstrap required, opened a new log
I20260811 06:29:43.215467 1247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:43.192794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51556 (local address 127.28.239.254:33329)
0811 06:29:43.193687 (+ 893us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:43.193705 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:43.193760 (+ 55us) server_negotiation.cc:407] Connection header received
0811 06:29:43.195809 (+ 2049us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:43.195843 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:43.196431 (+ 588us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:43.196893 (+ 462us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:43.198394 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:43.200449 (+ 2055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:43.201804 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:43.202899 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:43.203077 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:43.207633 (+ 4556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:29:43.207641 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:29:43.207657 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:29:43.207695 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:29:43.210964 (+ 3269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:43.211884 (+ 920us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:43.211890 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:43.211896 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:43.212083 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:43.212501 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:43.212504 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:43.212506 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:43.213058 (+ 552us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:29:43.213318 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:43.213861 (+ 543us) server_negotiation.cc:299] Negotiation successful
0811 06:29:43.214331 (+ 470us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":610,"thread_start_us":373,"threads_started":1}
I20260811 06:29:43.218119 1243 raft_consensus.cc:348] T 00000000000000000000000000000000 P b4f0cd7764634d19ae4ec8c39eb5df30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4f0cd7764634d19ae4ec8c39eb5df30" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 33329 } }
I20260811 06:29:43.218595 1243 raft_consensus.cc:374] T 00000000000000000000000000000000 P b4f0cd7764634d19ae4ec8c39eb5df30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:29:43.218737 1243 raft_consensus.cc:729] T 00000000000000000000000000000000 P b4f0cd7764634d19ae4ec8c39eb5df30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4f0cd7764634d19ae4ec8c39eb5df30, State: Initialized, Role: FOLLOWER
I20260811 06:29:43.219348 1243 consensus_queue.cc:260] T 00000000000000000000000000000000 P b4f0cd7764634d19ae4ec8c39eb5df30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4f0cd7764634d19ae4ec8c39eb5df30" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 33329 } }
I20260811 06:29:43.219591 1243 raft_consensus.cc:388] T 00000000000000000000000000000000 P b4f0cd7764634d19ae4ec8c39eb5df30 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:29:43.219732 1243 raft_consensus.cc:482] T 00000000000000000000000000000000 P b4f0cd7764634d19ae4ec8c39eb5df30 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:29:43.219909 1243 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b4f0cd7764634d19ae4ec8c39eb5df30 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:29:43.222254 1243 raft_consensus.cc:504] T 00000000000000000000000000000000 P b4f0cd7764634d19ae4ec8c39eb5df30 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4f0cd7764634d19ae4ec8c39eb5df30" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 33329 } }
I20260811 06:29:43.222783 1243 leader_election.cc:302] T 00000000000000000000000000000000 P b4f0cd7764634d19ae4ec8c39eb5df30 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b4f0cd7764634d19ae4ec8c39eb5df30; no voters:
I20260811 06:29:43.223460 1243 leader_election.cc:288] T 00000000000000000000000000000000 P b4f0cd7764634d19ae4ec8c39eb5df30 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:29:43.223572 1251 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b4f0cd7764634d19ae4ec8c39eb5df30 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:29:43.225018 1251 raft_consensus.cc:686] T 00000000000000000000000000000000 P b4f0cd7764634d19ae4ec8c39eb5df30 [term 1 LEADER]: Becoming Leader. State: Replica: b4f0cd7764634d19ae4ec8c39eb5df30, State: Running, Role: LEADER
I20260811 06:29:43.225574 1251 consensus_queue.cc:237] T 00000000000000000000000000000000 P b4f0cd7764634d19ae4ec8c39eb5df30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4f0cd7764634d19ae4ec8c39eb5df30" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 33329 } }
I20260811 06:29:43.225898 1243 sys_catalog.cc:564] T 00000000000000000000000000000000 P b4f0cd7764634d19ae4ec8c39eb5df30 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:29:43.228662 1256 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4f0cd7764634d19ae4ec8c39eb5df30 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b4f0cd7764634d19ae4ec8c39eb5df30. Latest consensus state: current_term: 1 leader_uuid: "b4f0cd7764634d19ae4ec8c39eb5df30" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4f0cd7764634d19ae4ec8c39eb5df30" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 33329 } } }
I20260811 06:29:43.229126 1256 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4f0cd7764634d19ae4ec8c39eb5df30 [sys.catalog]: This master's current role is: LEADER
I20260811 06:29:43.229094 1254 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4f0cd7764634d19ae4ec8c39eb5df30 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b4f0cd7764634d19ae4ec8c39eb5df30" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4f0cd7764634d19ae4ec8c39eb5df30" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 33329 } } }
I20260811 06:29:43.229637 1254 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4f0cd7764634d19ae4ec8c39eb5df30 [sys.catalog]: This master's current role is: LEADER
I20260811 06:29:43.232190 1272 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:29:43.236701 1272 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:29:43.245970 1272 catalog_manager.cc:1347] Generated new cluster ID: 8bf75d2db9404a86bb54dfd401a2a8ef
I20260811 06:29:43.246156 1272 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:29:43.266919 1272 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:29:43.268362 1272 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:29:43.276868 1272 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b4f0cd7764634d19ae4ec8c39eb5df30: Generated new TSK 0
I20260811 06:29:43.278071 1272 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.239.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.239.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.239.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.193.keytab.
Entry for principal kudu/127.28.239.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.193.keytab.
I20260811 06:29:43.341761 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.193:0
--local_ip_for_outbound_sockets=127.28.239.193
--webserver_interface=127.28.239.193
--webserver_port=0
--tserver_master_addrs=127.28.239.254:33329
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.193.keytab
--principal=kudu/127.28.239.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:43837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:29:43.694072 1291 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:29:43.694412 1291 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:29:43.694576 1291 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:29:43.702044 1291 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:29:43.702219 1291 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:29:43.702390 1291 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.193
I20260811 06:29:43.712716 1291 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:43837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.193.keytab
--principal=kudu/127.28.239.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.239.193
--webserver_port=0
--tserver_master_addrs=127.28.239.254:33329
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.193
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:29:43.714270 1291 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:29:43.715703 1291 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:29:43.730316 1296 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:29:43.730268 1303 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:29:43.730351 1297 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:29:43.731495 1291 server_base.cc:1034] running on GCE node
I20260811 06:29:43.732246 1291 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:29:43.733348 1291 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:29:43.734602 1291 hybrid_clock.cc:648] HybridClock initialized: now 1786429783734521 us; error 65 us; skew 500 ppm
I20260811 06:29:43.734951 1291 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:29:43 dist-test-slave-gkw7 krb5kdc[1103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429783, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:29:43.738330 1291 init.cc:377] Logged in from keytab as kudu/127.28.239.193@KRBTEST.COM (short username kudu)
I20260811 06:29:43.739920 1291 webserver.cc:460] Webserver started at http://127.28.239.193:43881/ using document root <none> and password file <none>
I20260811 06:29:43.740681 1291 fs_manager.cc:359] Metadata directory not provided
I20260811 06:29:43.740839 1291 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:29:43.741190 1291 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:29:43.744411 1291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-0/data/instance:
uuid: "7f0584f20015457493833985bb13a294"
format_stamp: "Formatted at 2026-08-11 06:29:43 on dist-test-slave-gkw7"
server_key: "827fa2849f7722b65a054a46a051e39a"
server_key_iv: "0b18c3f345660665a12fc4162fc3d233"
server_key_version: "encryptionkey@0"
I20260811 06:29:43.745304 1291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-0/wal/instance:
uuid: "7f0584f20015457493833985bb13a294"
format_stamp: "Formatted at 2026-08-11 06:29:43 on dist-test-slave-gkw7"
server_key: "827fa2849f7722b65a054a46a051e39a"
server_key_iv: "0b18c3f345660665a12fc4162fc3d233"
server_key_version: "encryptionkey@0"
I20260811 06:29:43.750798 1291 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 06:29:43.754555 1316 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:29:43.755982 1291 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 06:29:43.756192 1291 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-0/wal
uuid: "7f0584f20015457493833985bb13a294"
format_stamp: "Formatted at 2026-08-11 06:29:43 on dist-test-slave-gkw7"
server_key: "827fa2849f7722b65a054a46a051e39a"
server_key_iv: "0b18c3f345660665a12fc4162fc3d233"
server_key_version: "encryptionkey@0"
I20260811 06:29:43.756420 1291 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:29:43.774914 1291 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:29:43.779706 1291 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:29:43.780112 1291 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:29:43.781188 1291 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:29:43.782769 1291 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:29:43.782877 1291 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:29:43.783033 1291 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:29:43.783108 1291 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:29:43.818603 1291 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.193:42621
I20260811 06:29:43.818646 1491 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.193:42621 every 8 connection(s)
I20260811 06:29:43.820043 1291 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
I20260811 06:29:43.827783 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 1291
I20260811 06:29:43.828037 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-0/wal/instance
I20260811 06:29:43.828689 29631 external_mini_cluster.cc:1428] Setting key a85588aeb55d089c702f606c8a7bc9b0
Aug 11 06:29:43 dist-test-slave-gkw7 krb5kdc[1103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429783, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.193@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:29:43.842607 1495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:43.822876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.193:34149 (local address 127.28.239.254:33329)
0811 06:29:43.823383 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:43.823387 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:43.824292 (+ 905us) server_negotiation.cc:407] Connection header received
0811 06:29:43.826378 (+ 2086us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:43.826383 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:43.826485 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:43.826613 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:43.828439 (+ 1826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:43.829257 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:43.831447 (+ 2190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:43.831851 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:43.831954 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:43.836455 (+ 4501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:29:43.836460 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:29:43.836464 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:29:43.836498 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:29:43.839222 (+ 2724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:43.840047 (+ 825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:43.840057 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:43.840060 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:43.840184 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:43.840668 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:43.840672 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:43.840674 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:43.840956 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:29:43.841123 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:43.841981 (+ 858us) server_negotiation.cc:299] Negotiation successful
0811 06:29:43.842220 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":284,"threads_started":1}
I20260811 06:29:43.843644 1498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:43.823352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:33329 (local address 127.28.239.193:34149)
0811 06:29:43.824099 (+ 747us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:43.824153 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 06:29:43.826009 (+ 1856us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:29:43.826915 (+ 906us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:29:43.826924 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:29:43.827530 (+ 606us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:29:43.828260 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:43.828280 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:43.829451 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:43.829455 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:43.831280 (+ 1825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:43.831295 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:43.832098 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:43.832108 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:43.832464 (+ 356us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:43.833275 (+ 811us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:29:43.833317 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:29:43.836195 (+ 2878us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:29:43.839403 (+ 3208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:29:43.839413 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:29:43.839432 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:29:43.839901 (+ 469us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:29:43.840310 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:29:43.840315 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:29:43.840319 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:29:43.840530 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:29:43.841098 (+ 568us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:29:43.841107 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:29:43.841488 (+ 381us) client_negotiation.cc:769] Sending connection context
0811 06:29:43.841870 (+ 382us) client_negotiation.cc:240] Negotiation successful
0811 06:29:43.842450 (+ 580us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":307,"threads_started":1}
I20260811 06:29:43.845439 1492 heartbeater.cc:347] Connected to a master server at 127.28.239.254:33329
I20260811 06:29:43.845824 1492 heartbeater.cc:464] Registering TS with master...
I20260811 06:29:43.846612 1492 heartbeater.cc:511] Master 127.28.239.254:33329 requested a full tablet report, sending...
I20260811 06:29:43.849159 1185 ts_manager.cc:194] Registered new tserver with Master: 7f0584f20015457493833985bb13a294 (127.28.239.193:42621)
WARNING: no policy specified for kudu/127.28.239.194@KRBTEST.COM; defaulting to no policy
I20260811 06:29:43.851490 1185 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.239.193@KRBTEST.COM'} at 127.28.239.193:34149
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.239.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.239.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.194.keytab.
Entry for principal kudu/127.28.239.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.194.keytab.
I20260811 06:29:43.881224 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.194:0
--local_ip_for_outbound_sockets=127.28.239.194
--webserver_interface=127.28.239.194
--webserver_port=0
--tserver_master_addrs=127.28.239.254:33329
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.194.keytab
--principal=kudu/127.28.239.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:43837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:29:44.232467 1501 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:29:44.232779 1501 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:29:44.232884 1501 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:29:44.240375 1501 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:29:44.240515 1501 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:29:44.240633 1501 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.194
I20260811 06:29:44.250717 1501 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:43837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.194.keytab
--principal=kudu/127.28.239.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.239.194
--webserver_port=0
--tserver_master_addrs=127.28.239.254:33329
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.194
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:29:44.252133 1501 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:29:44.253508 1501 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:29:44.269006 1509 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:29:44.269024 1508 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:29:44.269109 1513 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:29:44.269917 1501 server_base.cc:1034] running on GCE node
I20260811 06:29:44.270646 1501 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:29:44.271766 1501 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:29:44.273054 1501 hybrid_clock.cc:648] HybridClock initialized: now 1786429784272953 us; error 72 us; skew 500 ppm
I20260811 06:29:44.273478 1501 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:29:44 dist-test-slave-gkw7 krb5kdc[1103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429784, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:29:44.283169 1501 init.cc:377] Logged in from keytab as kudu/127.28.239.194@KRBTEST.COM (short username kudu)
I20260811 06:29:44.284860 1501 webserver.cc:460] Webserver started at http://127.28.239.194:45693/ using document root <none> and password file <none>
I20260811 06:29:44.285722 1501 fs_manager.cc:359] Metadata directory not provided
I20260811 06:29:44.285869 1501 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:29:44.286222 1501 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:29:44.289460 1501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-1/data/instance:
uuid: "909e7f942c73428fa17dc07e197e943f"
format_stamp: "Formatted at 2026-08-11 06:29:44 on dist-test-slave-gkw7"
server_key: "9dba779aa462a32a873478b43b17f8fe"
server_key_iv: "4bb8480b4abe6755da09684195eedc36"
server_key_version: "encryptionkey@0"
I20260811 06:29:44.290472 1501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-1/wal/instance:
uuid: "909e7f942c73428fa17dc07e197e943f"
format_stamp: "Formatted at 2026-08-11 06:29:44 on dist-test-slave-gkw7"
server_key: "9dba779aa462a32a873478b43b17f8fe"
server_key_iv: "4bb8480b4abe6755da09684195eedc36"
server_key_version: "encryptionkey@0"
I20260811 06:29:44.296392 1501 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 06:29:44.300427 1525 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:29:44.302167 1501 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 06:29:44.302367 1501 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-1/wal
uuid: "909e7f942c73428fa17dc07e197e943f"
format_stamp: "Formatted at 2026-08-11 06:29:44 on dist-test-slave-gkw7"
server_key: "9dba779aa462a32a873478b43b17f8fe"
server_key_iv: "4bb8480b4abe6755da09684195eedc36"
server_key_version: "encryptionkey@0"
I20260811 06:29:44.302611 1501 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:29:44.329241 1501 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:29:44.335165 1501 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:29:44.335517 1501 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:29:44.336606 1501 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:29:44.338099 1501 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:29:44.338178 1501 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:29:44.338274 1501 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:29:44.338320 1501 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:29:44.373088 1501 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.194:39367
I20260811 06:29:44.373123 1691 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.194:39367 every 8 connection(s)
I20260811 06:29:44.374539 1501 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
I20260811 06:29:44.376466 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 1501
I20260811 06:29:44.376698 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-1/wal/instance
I20260811 06:29:44.377169 29631 external_mini_cluster.cc:1428] Setting key b7905db08e488900ad1e529e113dd2d4
Aug 11 06:29:44 dist-test-slave-gkw7 krb5kdc[1103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429784, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.194@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:29:44.397814 1694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:44.377688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.194:51403 (local address 127.28.239.254:33329)
0811 06:29:44.378571 (+ 883us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:44.378577 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:44.379538 (+ 961us) server_negotiation.cc:407] Connection header received
0811 06:29:44.381609 (+ 2071us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:44.381615 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:44.381713 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:44.381865 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:44.383454 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:44.384232 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:44.385894 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:44.386349 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:44.386484 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:44.390800 (+ 4316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:29:44.390805 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:29:44.390808 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:29:44.390844 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:29:44.394042 (+ 3198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:44.394955 (+ 913us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:44.394960 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:44.394963 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:44.395127 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:44.395709 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:44.395713 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:44.395716 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:44.396024 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:29:44.396232 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:44.397075 (+ 843us) server_negotiation.cc:299] Negotiation successful
0811 06:29:44.397349 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":742,"thread_start_us":610,"threads_started":1}
I20260811 06:29:44.398969 1695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:44.378444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:33329 (local address 127.28.239.194:51403)
0811 06:29:44.379360 (+ 916us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:44.379418 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 06:29:44.381303 (+ 1885us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:29:44.382112 (+ 809us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:29:44.382120 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:29:44.382659 (+ 539us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:29:44.383270 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:44.383289 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:44.384391 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:44.384396 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:44.385744 (+ 1348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:44.385757 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:44.386597 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:44.386605 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:44.386846 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:44.387657 (+ 811us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:29:44.387695 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:29:44.390547 (+ 2852us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:29:44.394225 (+ 3678us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:29:44.394240 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:29:44.394259 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:29:44.394791 (+ 532us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:29:44.395261 (+ 470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:29:44.395270 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:29:44.395277 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:29:44.395552 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:29:44.396221 (+ 669us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:29:44.396230 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:29:44.396636 (+ 406us) client_negotiation.cc:769] Sending connection context
0811 06:29:44.396954 (+ 318us) client_negotiation.cc:240] Negotiation successful
0811 06:29:44.397667 (+ 713us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":608,"thread_start_us":374,"threads_started":1}
WARNING: no policy specified for kudu/127.28.239.195@KRBTEST.COM; defaulting to no policy
I20260811 06:29:44.400647 1692 heartbeater.cc:347] Connected to a master server at 127.28.239.254:33329
I20260811 06:29:44.401024 1692 heartbeater.cc:464] Registering TS with master...
I20260811 06:29:44.401870 1692 heartbeater.cc:511] Master 127.28.239.254:33329 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.239.195@KRBTEST.COM" created.
I20260811 06:29:44.403687 1185 ts_manager.cc:194] Registered new tserver with Master: 909e7f942c73428fa17dc07e197e943f (127.28.239.194:39367)
I20260811 06:29:44.405041 1185 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.239.194@KRBTEST.COM'} at 127.28.239.194:51403
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.239.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.195.keytab.
Entry for principal kudu/127.28.239.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.195.keytab.
I20260811 06:29:44.431315 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.195:0
--local_ip_for_outbound_sockets=127.28.239.195
--webserver_interface=127.28.239.195
--webserver_port=0
--tserver_master_addrs=127.28.239.254:33329
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.195.keytab
--principal=kudu/127.28.239.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:43837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:29:44.789376 1699 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:29:44.789731 1699 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:29:44.789906 1699 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:29:44.797399 1699 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:29:44.797597 1699 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:29:44.797798 1699 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.195
I20260811 06:29:44.807888 1699 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:43837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.195.keytab
--principal=kudu/127.28.239.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.239.195
--webserver_port=0
--tserver_master_addrs=127.28.239.254:33329
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.195
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:29:44.809286 1699 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:29:44.810791 1699 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:29:44.824851 1706 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:29:44.825026 1704 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:29:44.825186 1709 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:29:44.825995 1699 server_base.cc:1034] running on GCE node
I20260811 06:29:44.826726 1699 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:29:44.827786 1699 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:29:44.829035 1699 hybrid_clock.cc:648] HybridClock initialized: now 1786429784828978 us; error 48 us; skew 500 ppm
I20260811 06:29:44.829501 1699 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:29:44 dist-test-slave-gkw7 krb5kdc[1103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429784, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:29:44.832934 1699 init.cc:377] Logged in from keytab as kudu/127.28.239.195@KRBTEST.COM (short username kudu)
I20260811 06:29:44.834530 1699 webserver.cc:460] Webserver started at http://127.28.239.195:35091/ using document root <none> and password file <none>
I20260811 06:29:44.835300 1699 fs_manager.cc:359] Metadata directory not provided
I20260811 06:29:44.835453 1699 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:29:44.835778 1699 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:29:44.838842 1699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-2/data/instance:
uuid: "7682da967fea4ccebcf340f6c2da5144"
format_stamp: "Formatted at 2026-08-11 06:29:44 on dist-test-slave-gkw7"
server_key: "1d488efa82f307b19656442145e1c820"
server_key_iv: "7c5e0e083f242bf3c478182be27f14d5"
server_key_version: "encryptionkey@0"
I20260811 06:29:44.839665 1699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-2/wal/instance:
uuid: "7682da967fea4ccebcf340f6c2da5144"
format_stamp: "Formatted at 2026-08-11 06:29:44 on dist-test-slave-gkw7"
server_key: "1d488efa82f307b19656442145e1c820"
server_key_iv: "7c5e0e083f242bf3c478182be27f14d5"
server_key_version: "encryptionkey@0"
I20260811 06:29:44.845099 1699 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 06:29:44.848830 1723 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:29:44.850400 1699 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:29:44.850625 1699 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-2/wal
uuid: "7682da967fea4ccebcf340f6c2da5144"
format_stamp: "Formatted at 2026-08-11 06:29:44 on dist-test-slave-gkw7"
server_key: "1d488efa82f307b19656442145e1c820"
server_key_iv: "7c5e0e083f242bf3c478182be27f14d5"
server_key_version: "encryptionkey@0"
I20260811 06:29:44.850860 1699 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:29:44.854604 1492 heartbeater.cc:503] Master 127.28.239.254:33329 was elected leader, sending a full tablet report...
I20260811 06:29:44.874164 1699 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:29:44.878726 1699 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:29:44.879101 1699 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:29:44.880201 1699 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:29:44.881750 1699 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:29:44.881911 1699 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:29:44.882088 1699 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:29:44.882172 1699 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:29:44.919595 1699 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.195:34153
I20260811 06:29:44.919660 1893 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.195:34153 every 8 connection(s)
I20260811 06:29:44.921167 1699 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
I20260811 06:29:44.926038 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 1699
I20260811 06:29:44.926259 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-2/wal/instance
I20260811 06:29:44.926729 29631 external_mini_cluster.cc:1428] Setting key 3762a4d0a8d92d9bbc7c6e0b6fcbe20a
Aug 11 06:29:44 dist-test-slave-gkw7 krb5kdc[1103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429784, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.195@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:29:44.943733 1902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:44.924037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.195:53729 (local address 127.28.239.254:33329)
0811 06:29:44.924575 (+ 538us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:44.924580 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:44.925420 (+ 840us) server_negotiation.cc:407] Connection header received
0811 06:29:44.927901 (+ 2481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:44.927907 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:44.928040 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:44.928178 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:44.930064 (+ 1886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:44.930998 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:44.932740 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:44.933146 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:44.933241 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:44.936985 (+ 3744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:29:44.936990 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:29:44.936993 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:29:44.937027 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:29:44.939804 (+ 2777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:44.940826 (+ 1022us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:44.940831 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:44.940834 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:44.940971 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:44.941494 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:44.941501 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:44.941504 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:44.941875 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:29:44.942067 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:44.943029 (+ 962us) server_negotiation.cc:299] Negotiation successful
0811 06:29:44.943288 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":418,"thread_start_us":300,"threads_started":1}
I20260811 06:29:44.944895 1903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:44.924452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:33329 (local address 127.28.239.195:53729)
0811 06:29:44.925232 (+ 780us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:44.925274 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 06:29:44.927591 (+ 2317us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:29:44.928462 (+ 871us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:29:44.928474 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:29:44.929097 (+ 623us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:29:44.929843 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:44.929866 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:44.931181 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:44.931185 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:44.932602 (+ 1417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:44.932615 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:44.933363 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:44.933368 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:44.933675 (+ 307us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:44.934367 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:29:44.934398 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:29:44.936763 (+ 2365us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:29:44.939963 (+ 3200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:29:44.939971 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:29:44.939985 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:29:44.940644 (+ 659us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:29:44.941146 (+ 502us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:29:44.941150 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:29:44.941152 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:29:44.941354 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:29:44.942024 (+ 670us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:29:44.942033 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:29:44.942428 (+ 395us) client_negotiation.cc:769] Sending connection context
0811 06:29:44.942737 (+ 309us) client_negotiation.cc:240] Negotiation successful
0811 06:29:44.943439 (+ 702us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":301,"threads_started":1}
I20260811 06:29:44.946460 1895 heartbeater.cc:347] Connected to a master server at 127.28.239.254:33329
I20260811 06:29:44.946797 1895 heartbeater.cc:464] Registering TS with master...
I20260811 06:29:44.947561 1895 heartbeater.cc:511] Master 127.28.239.254:33329 requested a full tablet report, sending...
I20260811 06:29:44.949076 1185 ts_manager.cc:194] Registered new tserver with Master: 7682da967fea4ccebcf340f6c2da5144 (127.28.239.195:34153)
I20260811 06:29:44.949975 1185 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.239.195@KRBTEST.COM'} at 127.28.239.195:53729
I20260811 06:29:44.955045 29631 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 06:29:44.993291 1902 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:29:44.990544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51564 (local address 127.28.239.254:33329)
0811 06:29:44.990744 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:44.990786 (+ 42us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:44.991289 (+ 503us) server_negotiation.cc:407] Connection header received
0811 06:29:44.991502 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:44.991507 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:44.991592 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:44.991706 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:44.993066 (+ 1360us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51564: BlockingRecv error: recv got EOF from 127.0.0.1:51564 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 06:29:44.996891 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 1291
W20260811 06:29:45.000114 29631 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 1291: Not authorized: Client connection negotiation failed: client connection to 127.28.239.193:42621: server requires authentication, but client does not have Kerberos credentials available
I20260811 06:29:45.000295 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 1291
I20260811 06:29:45.025992 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 1501
W20260811 06:29:45.028926 29631 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 1501: Not authorized: Client connection negotiation failed: client connection to 127.28.239.194:39367: server requires authentication, but client does not have Kerberos credentials available
I20260811 06:29:45.029100 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 1501
I20260811 06:29:45.054375 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 1699
W20260811 06:29:45.057979 29631 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 1699: Not authorized: Client connection negotiation failed: client connection to 127.28.239.195:34153: server requires authentication, but client does not have Kerberos credentials available
I20260811 06:29:45.058099 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 1699
I20260811 06:29:45.089354 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 1127
I20260811 06:29:45.090751 1232 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:29:45.196679 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 1127
2026-08-11T06:29:45Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (4764 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786429755927508-29631-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:29:45 dist-test-slave-gkw7 krb5kdc[1921](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:29:45 dist-test-slave-gkw7 krb5kdc[1921](info): set up 2 sockets
Aug 11 06:29:45 dist-test-slave-gkw7 krb5kdc[1921](info): commencing operation
krb5kdc: starting...
W20260811 06:29:46.535320 29631 mini_kdc.cc:121] Time spent starting KDC: real 1.288s user 0.001s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786429755927508-29631-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786429755927508-29631-0/krb5kdc/test-user.keytab.
Aug 11 06:29:46 dist-test-slave-gkw7 krb5kdc[1921](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429786, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:29:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:29:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.239.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.239.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.239.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab.
Entry for principal kudu/127.28.239.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab.
I20260811 06:29:46.764663 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786429755927508-29631-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786429755927508-29631-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786429755927508-29631-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:42137
--webserver_interface=127.28.239.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab
--principal=kudu/127.28.239.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:34517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.239.254:42137
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:29:47.104959 1947 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:29:47.105316 1947 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:29:47.105496 1947 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:29:47.112958 1947 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:29:47.113080 1947 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:29:47.113127 1947 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:29:47.113215 1947 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:29:47.113305 1947 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:29:47.123777 1947 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:34517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786429755927508-29631-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786429755927508-29631-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.239.254:42137
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab
--principal=kudu/127.28.239.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:42137
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.239.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786429755927508-29631-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:29:47.125198 1947 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:29:47.126674 1947 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:29:47.139066 1955 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:29:47.139400 1953 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:29:47.139889 1959 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:29:47.140879 1947 server_base.cc:1034] running on GCE node
I20260811 06:29:47.141800 1947 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:29:47.143306 1947 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:29:47.144515 1947 hybrid_clock.cc:648] HybridClock initialized: now 1786429787144455 us; error 57 us; skew 500 ppm
I20260811 06:29:47.144898 1947 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:29:47 dist-test-slave-gkw7 krb5kdc[1921](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429787, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:29:47.148311 1947 init.cc:377] Logged in from keytab as kudu/127.28.239.254@KRBTEST.COM (short username kudu)
I20260811 06:29:47.149996 1947 webserver.cc:460] Webserver started at http://127.28.239.254:45103/ using document root <none> and password file <none>
I20260811 06:29:47.150810 1947 fs_manager.cc:359] Metadata directory not provided
I20260811 06:29:47.150969 1947 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:29:47.151337 1947 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:29:47.154749 1947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786429755927508-29631-0/minicluster-data/master-0/data/instance:
uuid: "faf1114540f64a1c9690bd96d77a0c8b"
format_stamp: "Formatted at 2026-08-11 06:29:47 on dist-test-slave-gkw7"
server_key: "72795a7a14e56753f542173206df5564"
server_key_iv: "350dae8ca2008313670c5934f6f0daa9"
server_key_version: "encryptionkey@0"
I20260811 06:29:47.155648 1947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786429755927508-29631-0/minicluster-data/master-0/wal/instance:
uuid: "faf1114540f64a1c9690bd96d77a0c8b"
format_stamp: "Formatted at 2026-08-11 06:29:47 on dist-test-slave-gkw7"
server_key: "72795a7a14e56753f542173206df5564"
server_key_iv: "350dae8ca2008313670c5934f6f0daa9"
server_key_version: "encryptionkey@0"
I20260811 06:29:47.161372 1947 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 06:29:47.165146 1969 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:29:47.166560 1947 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:29:47.166699 1947 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786429755927508-29631-0/minicluster-data/master-0/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786429755927508-29631-0/minicluster-data/master-0/wal
uuid: "faf1114540f64a1c9690bd96d77a0c8b"
format_stamp: "Formatted at 2026-08-11 06:29:47 on dist-test-slave-gkw7"
server_key: "72795a7a14e56753f542173206df5564"
server_key_iv: "350dae8ca2008313670c5934f6f0daa9"
server_key_version: "encryptionkey@0"
I20260811 06:29:47.166877 1947 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786429755927508-29631-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786429755927508-29631-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786429755927508-29631-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:29:47.179989 1947 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:29:47.183974 1947 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:29:47.184218 1947 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:29:47.203323 1947 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.254:42137
I20260811 06:29:47.203358 2052 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.254:42137 every 8 connection(s)
I20260811 06:29:47.204843 1947 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
I20260811 06:29:47.207412 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 1947
I20260811 06:29:47.207667 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786429755927508-29631-0/minicluster-data/master-0/wal/instance
I20260811 06:29:47.208499 29631 external_mini_cluster.cc:1428] Setting key 585370503ecf4d79df683d182cf57f4e
I20260811 06:29:47.209568 2054 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:29:47.221486 2054 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P faf1114540f64a1c9690bd96d77a0c8b: Bootstrap starting.
Aug 11 06:29:47 dist-test-slave-gkw7 krb5kdc[1921](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429786, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:29:47.224915 2054 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P faf1114540f64a1c9690bd96d77a0c8b: Neither blocks nor log segments found. Creating new log.
I20260811 06:29:47.226403 2054 log.cc:824] T 00000000000000000000000000000000 P faf1114540f64a1c9690bd96d77a0c8b: Log is configured to *not* fsync() on all Append() calls
I20260811 06:29:47.229996 2054 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P faf1114540f64a1c9690bd96d77a0c8b: No bootstrap required, opened a new log
I20260811 06:29:47.232081 2060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:47.210460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57246 (local address 127.28.239.254:42137)
0811 06:29:47.211250 (+ 790us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:47.211263 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:47.211307 (+ 44us) server_negotiation.cc:407] Connection header received
0811 06:29:47.212936 (+ 1629us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:47.212958 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:47.213358 (+ 400us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:47.213824 (+ 466us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:47.215308 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:47.216865 (+ 1557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:47.218189 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:47.218922 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:47.219134 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:47.223344 (+ 4210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:29:47.223352 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:29:47.223370 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:29:47.223407 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:29:47.227217 (+ 3810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:47.228117 (+ 900us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:47.228127 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:47.228134 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:47.228351 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:47.228708 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:47.228713 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:47.228716 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:47.229459 (+ 743us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:29:47.229794 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:47.230201 (+ 407us) server_negotiation.cc:299] Negotiation successful
0811 06:29:47.230684 (+ 483us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":582,"thread_start_us":358,"threads_started":1}
I20260811 06:29:47.236287 29631 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 06:29:47.239084 2054 raft_consensus.cc:348] T 00000000000000000000000000000000 P faf1114540f64a1c9690bd96d77a0c8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "faf1114540f64a1c9690bd96d77a0c8b" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 42137 } }
I20260811 06:29:47.239500 2054 raft_consensus.cc:374] T 00000000000000000000000000000000 P faf1114540f64a1c9690bd96d77a0c8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:29:47.239629 2054 raft_consensus.cc:729] T 00000000000000000000000000000000 P faf1114540f64a1c9690bd96d77a0c8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: faf1114540f64a1c9690bd96d77a0c8b, State: Initialized, Role: FOLLOWER
I20260811 06:29:47.240213 2054 consensus_queue.cc:260] T 00000000000000000000000000000000 P faf1114540f64a1c9690bd96d77a0c8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "faf1114540f64a1c9690bd96d77a0c8b" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 42137 } }
I20260811 06:29:47.240458 2054 raft_consensus.cc:388] T 00000000000000000000000000000000 P faf1114540f64a1c9690bd96d77a0c8b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:29:47.240597 2054 raft_consensus.cc:482] T 00000000000000000000000000000000 P faf1114540f64a1c9690bd96d77a0c8b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:29:47.240785 2054 raft_consensus.cc:3037] T 00000000000000000000000000000000 P faf1114540f64a1c9690bd96d77a0c8b [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:29:47.243089 2054 raft_consensus.cc:504] T 00000000000000000000000000000000 P faf1114540f64a1c9690bd96d77a0c8b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "faf1114540f64a1c9690bd96d77a0c8b" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 42137 } }
I20260811 06:29:47.243628 2054 leader_election.cc:302] T 00000000000000000000000000000000 P faf1114540f64a1c9690bd96d77a0c8b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: faf1114540f64a1c9690bd96d77a0c8b; no voters:
I20260811 06:29:47.244338 2054 leader_election.cc:288] T 00000000000000000000000000000000 P faf1114540f64a1c9690bd96d77a0c8b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:29:47.244432 2067 raft_consensus.cc:2781] T 00000000000000000000000000000000 P faf1114540f64a1c9690bd96d77a0c8b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:29:47.245680 2067 raft_consensus.cc:686] T 00000000000000000000000000000000 P faf1114540f64a1c9690bd96d77a0c8b [term 1 LEADER]: Becoming Leader. State: Replica: faf1114540f64a1c9690bd96d77a0c8b, State: Running, Role: LEADER
I20260811 06:29:47.246338 2067 consensus_queue.cc:237] T 00000000000000000000000000000000 P faf1114540f64a1c9690bd96d77a0c8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "faf1114540f64a1c9690bd96d77a0c8b" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 42137 } }
I20260811 06:29:47.246910 2054 sys_catalog.cc:564] T 00000000000000000000000000000000 P faf1114540f64a1c9690bd96d77a0c8b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:29:47.249614 2069 sys_catalog.cc:455] T 00000000000000000000000000000000 P faf1114540f64a1c9690bd96d77a0c8b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "faf1114540f64a1c9690bd96d77a0c8b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "faf1114540f64a1c9690bd96d77a0c8b" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 42137 } } }
I20260811 06:29:47.250105 2069 sys_catalog.cc:458] T 00000000000000000000000000000000 P faf1114540f64a1c9690bd96d77a0c8b [sys.catalog]: This master's current role is: LEADER
I20260811 06:29:47.249583 2070 sys_catalog.cc:455] T 00000000000000000000000000000000 P faf1114540f64a1c9690bd96d77a0c8b [sys.catalog]: SysCatalogTable state changed. Reason: New leader faf1114540f64a1c9690bd96d77a0c8b. Latest consensus state: current_term: 1 leader_uuid: "faf1114540f64a1c9690bd96d77a0c8b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "faf1114540f64a1c9690bd96d77a0c8b" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 42137 } } }
I20260811 06:29:47.250497 2070 sys_catalog.cc:458] T 00000000000000000000000000000000 P faf1114540f64a1c9690bd96d77a0c8b [sys.catalog]: This master's current role is: LEADER
I20260811 06:29:47.257257 2079 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:29:47.264344 2079 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:29:47.293859 2079 catalog_manager.cc:1347] Generated new cluster ID: 3ad652a27a7743af8f776a01c07d7f1c
I20260811 06:29:47.293987 2079 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:29:47.306630 2060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:47.290620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57262 (local address 127.28.239.254:42137)
0811 06:29:47.290798 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:47.290806 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:47.297654 (+ 6848us) server_negotiation.cc:407] Connection header received
0811 06:29:47.298151 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:47.298158 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:47.298247 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:47.298338 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:47.299707 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:47.300769 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:47.302629 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:47.303315 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:47.303403 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:47.304862 (+ 1459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:29:47.304870 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:29:47.304875 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:29:47.305803 (+ 928us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:29:47.305862 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:29:47.305888 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:29:47.305960 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:47.306186 (+ 226us) server_negotiation.cc:299] Negotiation successful
0811 06:29:47.306293 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 06:29:47.326586 2079 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:29:47.328099 2079 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:29:47.336694 2079 catalog_manager.cc:5868] T 00000000000000000000000000000000 P faf1114540f64a1c9690bd96d77a0c8b: Generated new TSK 0
I20260811 06:29:47.337809 2079 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:29:47.353053 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 1947
I20260811 06:29:47.354549 2042 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:29:47.448182 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 1947
2026-08-11T06:29:47Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (2248 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:29:47 dist-test-slave-gkw7 krb5kdc[2110](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:29:47 dist-test-slave-gkw7 krb5kdc[2110](info): set up 2 sockets
Aug 11 06:29:47 dist-test-slave-gkw7 krb5kdc[2110](info): commencing operation
krb5kdc: starting...
W20260811 06:29:48.814392 29631 mini_kdc.cc:121] Time spent starting KDC: real 1.319s user 0.003s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/krb5kdc/test-user.keytab.
Aug 11 06:29:48 dist-test-slave-gkw7 krb5kdc[2110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429788, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:29:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:29:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.239.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.239.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.239.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab.
Entry for principal kudu/127.28.239.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab.
I20260811 06:29:49.041234 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:41475
--webserver_interface=127.28.239.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab
--principal=kudu/127.28.239.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:45937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.239.254:41475
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:29:49.407369 2129 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:29:49.407691 2129 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:29:49.407888 2129 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:29:49.415385 2129 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:29:49.415508 2129 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:29:49.415624 2129 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:29:49.415740 2129 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:29:49.415822 2129 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:29:49.426478 2129 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:45937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.239.254:41475
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab
--principal=kudu/127.28.239.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:41475
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.239.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:29:49.427945 2129 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:29:49.429358 2129 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:29:49.442248 2141 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:29:49.442224 2137 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:29:49.442938 2136 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:29:49.443351 2129 server_base.cc:1034] running on GCE node
I20260811 06:29:49.444214 2129 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:29:49.445789 2129 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:29:49.447010 2129 hybrid_clock.cc:648] HybridClock initialized: now 1786429789446957 us; error 62 us; skew 500 ppm
I20260811 06:29:49.447386 2129 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:29:49 dist-test-slave-gkw7 krb5kdc[2110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429789, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:29:49.455994 2129 init.cc:377] Logged in from keytab as kudu/127.28.239.254@KRBTEST.COM (short username kudu)
I20260811 06:29:49.457762 2129 webserver.cc:460] Webserver started at http://127.28.239.254:46153/ using document root <none> and password file <none>
I20260811 06:29:49.458528 2129 fs_manager.cc:359] Metadata directory not provided
I20260811 06:29:49.458684 2129 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:29:49.459034 2129 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:29:49.462314 2129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/minicluster-data/master-0/data/instance:
uuid: "32d18c40455547e6b79cd75affc671c0"
format_stamp: "Formatted at 2026-08-11 06:29:49 on dist-test-slave-gkw7"
server_key: "8008e1d7b8a11c6131dab56a764dd9b9"
server_key_iv: "b5bf4633048ff25976236cae221b7668"
server_key_version: "encryptionkey@0"
I20260811 06:29:49.463121 2129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/minicluster-data/master-0/wal/instance:
uuid: "32d18c40455547e6b79cd75affc671c0"
format_stamp: "Formatted at 2026-08-11 06:29:49 on dist-test-slave-gkw7"
server_key: "8008e1d7b8a11c6131dab56a764dd9b9"
server_key_iv: "b5bf4633048ff25976236cae221b7668"
server_key_version: "encryptionkey@0"
I20260811 06:29:49.468909 2129 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 06:29:49.472957 2159 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:29:49.474586 2129 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:29:49.474781 2129 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/minicluster-data/master-0/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/minicluster-data/master-0/wal
uuid: "32d18c40455547e6b79cd75affc671c0"
format_stamp: "Formatted at 2026-08-11 06:29:49 on dist-test-slave-gkw7"
server_key: "8008e1d7b8a11c6131dab56a764dd9b9"
server_key_iv: "b5bf4633048ff25976236cae221b7668"
server_key_version: "encryptionkey@0"
I20260811 06:29:49.474994 2129 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:29:49.497462 2129 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:29:49.502707 2129 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:29:49.503034 2129 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:29:49.522821 2129 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.254:41475
I20260811 06:29:49.522866 2230 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.254:41475 every 8 connection(s)
I20260811 06:29:49.524281 2129 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
I20260811 06:29:49.528213 2231 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:29:49.528800 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 2129
I20260811 06:29:49.529011 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/minicluster-data/master-0/wal/instance
I20260811 06:29:49.529767 29631 external_mini_cluster.cc:1428] Setting key aa22cbfd928b364b1bf09f405c67f393
I20260811 06:29:49.543076 2231 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32d18c40455547e6b79cd75affc671c0: Bootstrap starting.
Aug 11 06:29:49 dist-test-slave-gkw7 krb5kdc[2110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429788, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:29:49.546603 2231 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 32d18c40455547e6b79cd75affc671c0: Neither blocks nor log segments found. Creating new log.
I20260811 06:29:49.548121 2231 log.cc:824] T 00000000000000000000000000000000 P 32d18c40455547e6b79cd75affc671c0: Log is configured to *not* fsync() on all Append() calls
I20260811 06:29:49.551688 2231 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32d18c40455547e6b79cd75affc671c0: No bootstrap required, opened a new log
I20260811 06:29:49.555388 2237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:49.532057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36766 (local address 127.28.239.254:41475)
0811 06:29:49.532961 (+ 904us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:49.532983 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:49.533043 (+ 60us) server_negotiation.cc:407] Connection header received
0811 06:29:49.535149 (+ 2106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:49.535180 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:49.535723 (+ 543us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:49.536226 (+ 503us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:49.537568 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:49.539700 (+ 2132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:49.541012 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:49.542128 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:49.542320 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:49.546500 (+ 4180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:29:49.546513 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:29:49.546534 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:29:49.546598 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:29:49.550597 (+ 3999us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:49.551326 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:49.551334 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:49.551342 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:49.551558 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:49.551980 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:49.551984 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:49.551990 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:49.552755 (+ 765us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:29:49.553095 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:49.553376 (+ 281us) server_negotiation.cc:299] Negotiation successful
0811 06:29:49.553899 (+ 523us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":586,"thread_start_us":357,"threads_started":1}
I20260811 06:29:49.561161 2231 raft_consensus.cc:348] T 00000000000000000000000000000000 P 32d18c40455547e6b79cd75affc671c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32d18c40455547e6b79cd75affc671c0" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 41475 } }
I20260811 06:29:49.561568 2231 raft_consensus.cc:374] T 00000000000000000000000000000000 P 32d18c40455547e6b79cd75affc671c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:29:49.561681 2231 raft_consensus.cc:729] T 00000000000000000000000000000000 P 32d18c40455547e6b79cd75affc671c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32d18c40455547e6b79cd75affc671c0, State: Initialized, Role: FOLLOWER
I20260811 06:29:49.562265 2231 consensus_queue.cc:260] T 00000000000000000000000000000000 P 32d18c40455547e6b79cd75affc671c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32d18c40455547e6b79cd75affc671c0" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 41475 } }
I20260811 06:29:49.562484 2231 raft_consensus.cc:388] T 00000000000000000000000000000000 P 32d18c40455547e6b79cd75affc671c0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:29:49.562588 2231 raft_consensus.cc:482] T 00000000000000000000000000000000 P 32d18c40455547e6b79cd75affc671c0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:29:49.562704 2231 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 32d18c40455547e6b79cd75affc671c0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:29:49.565248 2231 raft_consensus.cc:504] T 00000000000000000000000000000000 P 32d18c40455547e6b79cd75affc671c0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32d18c40455547e6b79cd75affc671c0" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 41475 } }
I20260811 06:29:49.565793 2231 leader_election.cc:302] T 00000000000000000000000000000000 P 32d18c40455547e6b79cd75affc671c0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 32d18c40455547e6b79cd75affc671c0; no voters:
I20260811 06:29:49.566388 2231 leader_election.cc:288] T 00000000000000000000000000000000 P 32d18c40455547e6b79cd75affc671c0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:29:49.566547 2243 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 32d18c40455547e6b79cd75affc671c0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:29:49.567248 2243 raft_consensus.cc:686] T 00000000000000000000000000000000 P 32d18c40455547e6b79cd75affc671c0 [term 1 LEADER]: Becoming Leader. State: Replica: 32d18c40455547e6b79cd75affc671c0, State: Running, Role: LEADER
I20260811 06:29:49.567780 2243 consensus_queue.cc:237] T 00000000000000000000000000000000 P 32d18c40455547e6b79cd75affc671c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32d18c40455547e6b79cd75affc671c0" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 41475 } }
I20260811 06:29:49.568539 2231 sys_catalog.cc:564] T 00000000000000000000000000000000 P 32d18c40455547e6b79cd75affc671c0 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:29:49.570956 2247 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32d18c40455547e6b79cd75affc671c0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 32d18c40455547e6b79cd75affc671c0. Latest consensus state: current_term: 1 leader_uuid: "32d18c40455547e6b79cd75affc671c0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32d18c40455547e6b79cd75affc671c0" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 41475 } } }
I20260811 06:29:49.571324 2247 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32d18c40455547e6b79cd75affc671c0 [sys.catalog]: This master's current role is: LEADER
I20260811 06:29:49.571964 2246 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32d18c40455547e6b79cd75affc671c0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "32d18c40455547e6b79cd75affc671c0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32d18c40455547e6b79cd75affc671c0" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 41475 } } }
I20260811 06:29:49.572461 2246 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32d18c40455547e6b79cd75affc671c0 [sys.catalog]: This master's current role is: LEADER
I20260811 06:29:49.575054 2252 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:29:49.580976 2252 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:29:49.590451 2252 catalog_manager.cc:1347] Generated new cluster ID: 2379300a283e48fea22984d891927ca0
I20260811 06:29:49.590584 2252 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:29:49.597827 2252 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:29:49.599123 2252 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:29:49.607883 2252 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 32d18c40455547e6b79cd75affc671c0: Generated new TSK 0
I20260811 06:29:49.608777 2252 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.239.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.239.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.239.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.193.keytab.
Entry for principal kudu/127.28.239.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.193.keytab.
I20260811 06:29:49.681082 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.193:0
--local_ip_for_outbound_sockets=127.28.239.193
--webserver_interface=127.28.239.193
--webserver_port=0
--tserver_master_addrs=127.28.239.254:41475
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.193.keytab
--principal=kudu/127.28.239.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:45937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:29:50.049396 2277 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:29:50.049768 2277 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:29:50.049942 2277 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:29:50.057475 2277 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:29:50.057667 2277 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:29:50.057859 2277 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.193
I20260811 06:29:50.067938 2277 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:45937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.193.keytab
--principal=kudu/127.28.239.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.239.193
--webserver_port=0
--tserver_master_addrs=127.28.239.254:41475
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.193
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:29:50.069471 2277 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:29:50.070899 2277 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:29:50.085067 2283 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:29:50.085565 2282 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:29:50.085666 2286 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:29:50.086175 2277 server_base.cc:1034] running on GCE node
I20260811 06:29:50.086803 2277 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:29:50.087781 2277 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:29:50.089015 2277 hybrid_clock.cc:648] HybridClock initialized: now 1786429790088971 us; error 59 us; skew 500 ppm
I20260811 06:29:50.089361 2277 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:29:50 dist-test-slave-gkw7 krb5kdc[2110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429790, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:29:50.092604 2277 init.cc:377] Logged in from keytab as kudu/127.28.239.193@KRBTEST.COM (short username kudu)
I20260811 06:29:50.094172 2277 webserver.cc:460] Webserver started at http://127.28.239.193:33749/ using document root <none> and password file <none>
I20260811 06:29:50.094907 2277 fs_manager.cc:359] Metadata directory not provided
I20260811 06:29:50.095014 2277 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:29:50.095292 2277 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:29:50.098428 2277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-0/data/instance:
uuid: "f8511e5709a043a78d849e22604248c7"
format_stamp: "Formatted at 2026-08-11 06:29:50 on dist-test-slave-gkw7"
server_key: "9dba6dece8d5675dff65689b9b826cab"
server_key_iv: "79eb3e1c63f614c3fcd82e5ddb14eac9"
server_key_version: "encryptionkey@0"
I20260811 06:29:50.099196 2277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-0/wal/instance:
uuid: "f8511e5709a043a78d849e22604248c7"
format_stamp: "Formatted at 2026-08-11 06:29:50 on dist-test-slave-gkw7"
server_key: "9dba6dece8d5675dff65689b9b826cab"
server_key_iv: "79eb3e1c63f614c3fcd82e5ddb14eac9"
server_key_version: "encryptionkey@0"
I20260811 06:29:50.104487 2277 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 06:29:50.108322 2298 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:29:50.109781 2277 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 06:29:50.109954 2277 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-0/wal
uuid: "f8511e5709a043a78d849e22604248c7"
format_stamp: "Formatted at 2026-08-11 06:29:50 on dist-test-slave-gkw7"
server_key: "9dba6dece8d5675dff65689b9b826cab"
server_key_iv: "79eb3e1c63f614c3fcd82e5ddb14eac9"
server_key_version: "encryptionkey@0"
I20260811 06:29:50.110181 2277 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:29:50.142930 2277 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:29:50.147374 2277 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:29:50.147706 2277 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:29:50.148801 2277 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:29:50.150240 2277 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:29:50.150352 2277 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:29:50.150513 2277 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:29:50.150588 2277 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:29:50.185799 2277 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.193:43337
I20260811 06:29:50.185846 2480 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.193:43337 every 8 connection(s)
I20260811 06:29:50.187182 2277 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
I20260811 06:29:50.188321 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 2277
I20260811 06:29:50.188541 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-0/wal/instance
I20260811 06:29:50.189460 29631 external_mini_cluster.cc:1428] Setting key b79047c6c2ff4d77d54f42b1b1a84681
Aug 11 06:29:50 dist-test-slave-gkw7 krb5kdc[2110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429790, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.193@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:29:50.211691 2486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:50.190425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.193:50729 (local address 127.28.239.254:41475)
0811 06:29:50.191137 (+ 712us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:50.191142 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:50.192308 (+ 1166us) server_negotiation.cc:407] Connection header received
0811 06:29:50.195378 (+ 3070us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:50.195386 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:50.195475 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:50.195588 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:50.197838 (+ 2250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:50.198908 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:50.201136 (+ 2228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:50.201894 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:50.202018 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:50.205574 (+ 3556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:29:50.205581 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:29:50.205586 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:29:50.205630 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:29:50.208291 (+ 2661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:50.209177 (+ 886us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:50.209183 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:50.209188 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:50.209347 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:50.209857 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:50.209863 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:50.209867 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:50.210146 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:29:50.210317 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:50.211056 (+ 739us) server_negotiation.cc:299] Negotiation successful
0811 06:29:50.211302 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":575,"thread_start_us":451,"threads_started":1}
WARNING: no policy specified for kudu/127.28.239.194@KRBTEST.COM; defaulting to no policy
I20260811 06:29:50.212765 2489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:50.191188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:41475 (local address 127.28.239.193:50729)
0811 06:29:50.192109 (+ 921us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:50.192178 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0811 06:29:50.195009 (+ 2831us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:29:50.195893 (+ 884us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:29:50.195908 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:29:50.196722 (+ 814us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:29:50.197622 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:50.197652 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:50.199087 (+ 1435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:50.199096 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:50.200973 (+ 1877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:50.200986 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:50.202121 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:50.202126 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:50.202325 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:50.203052 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:29:50.203075 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:29:50.205328 (+ 2253us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:29:50.208450 (+ 3122us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:29:50.208460 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:29:50.208477 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:29:50.209046 (+ 569us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:29:50.209468 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:29:50.209473 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:29:50.209477 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:29:50.209734 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:29:50.210290 (+ 556us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:29:50.210298 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:29:50.210659 (+ 361us) client_negotiation.cc:769] Sending connection context
0811 06:29:50.210947 (+ 288us) client_negotiation.cc:240] Negotiation successful
0811 06:29:50.211533 (+ 586us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":592,"thread_start_us":388,"threads_started":1}
I20260811 06:29:50.214485 2481 heartbeater.cc:347] Connected to a master server at 127.28.239.254:41475
I20260811 06:29:50.214818 2481 heartbeater.cc:464] Registering TS with master...
I20260811 06:29:50.215644 2481 heartbeater.cc:511] Master 127.28.239.254:41475 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.239.194@KRBTEST.COM" created.
I20260811 06:29:50.218205 2181 ts_manager.cc:194] Registered new tserver with Master: f8511e5709a043a78d849e22604248c7 (127.28.239.193:43337)
I20260811 06:29:50.220396 2181 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.239.193@KRBTEST.COM'} at 127.28.239.193:50729
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.239.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.194.keytab.
Entry for principal kudu/127.28.239.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.194.keytab.
I20260811 06:29:50.242691 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.194:0
--local_ip_for_outbound_sockets=127.28.239.194
--webserver_interface=127.28.239.194
--webserver_port=0
--tserver_master_addrs=127.28.239.254:41475
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.194.keytab
--principal=kudu/127.28.239.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:45937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:29:50.598889 2494 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:29:50.599206 2494 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:29:50.599305 2494 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:29:50.606775 2494 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:29:50.606904 2494 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:29:50.607023 2494 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.194
I20260811 06:29:50.619099 2494 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:45937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.194.keytab
--principal=kudu/127.28.239.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.239.194
--webserver_port=0
--tserver_master_addrs=127.28.239.254:41475
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.194
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:29:50.620548 2494 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:29:50.622135 2494 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:29:50.636796 2500 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:29:50.636826 2501 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:29:50.636750 2504 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:29:50.638166 2494 server_base.cc:1034] running on GCE node
I20260811 06:29:50.638911 2494 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:29:50.639895 2494 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:29:50.641147 2494 hybrid_clock.cc:648] HybridClock initialized: now 1786429790641099 us; error 91 us; skew 500 ppm
I20260811 06:29:50.641499 2494 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:29:50 dist-test-slave-gkw7 krb5kdc[2110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429790, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:29:50.644928 2494 init.cc:377] Logged in from keytab as kudu/127.28.239.194@KRBTEST.COM (short username kudu)
I20260811 06:29:50.646512 2494 webserver.cc:460] Webserver started at http://127.28.239.194:34383/ using document root <none> and password file <none>
I20260811 06:29:50.647271 2494 fs_manager.cc:359] Metadata directory not provided
I20260811 06:29:50.647428 2494 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:29:50.647749 2494 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:29:50.650940 2494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-1/data/instance:
uuid: "c2919086d12a47519ac09f54bf822a06"
format_stamp: "Formatted at 2026-08-11 06:29:50 on dist-test-slave-gkw7"
server_key: "e43afff827748b1442b6efbd0f09e3e2"
server_key_iv: "287116cb2d655bab111f9c456649a84e"
server_key_version: "encryptionkey@0"
I20260811 06:29:50.651822 2494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-1/wal/instance:
uuid: "c2919086d12a47519ac09f54bf822a06"
format_stamp: "Formatted at 2026-08-11 06:29:50 on dist-test-slave-gkw7"
server_key: "e43afff827748b1442b6efbd0f09e3e2"
server_key_iv: "287116cb2d655bab111f9c456649a84e"
server_key_version: "encryptionkey@0"
I20260811 06:29:50.657466 2494 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 06:29:50.661144 2521 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:29:50.662673 2494 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:29:50.662879 2494 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-1/wal
uuid: "c2919086d12a47519ac09f54bf822a06"
format_stamp: "Formatted at 2026-08-11 06:29:50 on dist-test-slave-gkw7"
server_key: "e43afff827748b1442b6efbd0f09e3e2"
server_key_iv: "287116cb2d655bab111f9c456649a84e"
server_key_version: "encryptionkey@0"
I20260811 06:29:50.663098 2494 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:29:50.682996 2494 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:29:50.687297 2494 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:29:50.687657 2494 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:29:50.688700 2494 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:29:50.690207 2494 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:29:50.690327 2494 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:29:50.690490 2494 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:29:50.690565 2494 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:29:50.725793 2494 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.194:38767
I20260811 06:29:50.725822 2697 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.194:38767 every 8 connection(s)
I20260811 06:29:50.727248 2494 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
I20260811 06:29:50.736552 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 2494
I20260811 06:29:50.736768 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-1/wal/instance
I20260811 06:29:50.737260 29631 external_mini_cluster.cc:1428] Setting key ce10d5d20d5ea13e689cc5972523c9c8
Aug 11 06:29:50 dist-test-slave-gkw7 krb5kdc[2110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429790, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.194@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:29:50.749378 2702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:50.730021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.194:40285 (local address 127.28.239.254:41475)
0811 06:29:50.730508 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:50.730516 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:50.731589 (+ 1073us) server_negotiation.cc:407] Connection header received
0811 06:29:50.733391 (+ 1802us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:50.733396 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:50.733466 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:50.733634 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:50.735221 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:50.735957 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:50.737742 (+ 1785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:50.738250 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:50.738361 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:50.742206 (+ 3845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:29:50.742215 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:29:50.742223 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:29:50.742270 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:29:50.746016 (+ 3746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:50.746824 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:50.746833 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:50.746839 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:50.747010 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:50.747500 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:50.747503 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:50.747505 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:50.747812 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:29:50.747936 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:50.748928 (+ 992us) server_negotiation.cc:299] Negotiation successful
0811 06:29:50.749103 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":223,"threads_started":1}
I20260811 06:29:50.751015 2704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:50.730682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:41475 (local address 127.28.239.194:40285)
0811 06:29:50.731408 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:50.731452 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 06:29:50.733129 (+ 1677us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:29:50.733838 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:29:50.733846 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:29:50.734376 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:29:50.734972 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:50.734990 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:50.736096 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:50.736100 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:50.737579 (+ 1479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:50.737601 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:50.738500 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:50.738507 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:50.738710 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:50.739604 (+ 894us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:29:50.739633 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:29:50.741941 (+ 2308us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:29:50.746189 (+ 4248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:29:50.746197 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:29:50.746213 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:29:50.746670 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:29:50.747125 (+ 455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:29:50.747128 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:29:50.747130 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:29:50.747370 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:29:50.747960 (+ 590us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:29:50.747970 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:29:50.748484 (+ 514us) client_negotiation.cc:769] Sending connection context
0811 06:29:50.748862 (+ 378us) client_negotiation.cc:240] Negotiation successful
0811 06:29:50.749500 (+ 638us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":275,"threads_started":1}
I20260811 06:29:50.752532 2700 heartbeater.cc:347] Connected to a master server at 127.28.239.254:41475
I20260811 06:29:50.752866 2700 heartbeater.cc:464] Registering TS with master...
I20260811 06:29:50.753723 2700 heartbeater.cc:511] Master 127.28.239.254:41475 requested a full tablet report, sending...
I20260811 06:29:50.755546 2181 ts_manager.cc:194] Registered new tserver with Master: c2919086d12a47519ac09f54bf822a06 (127.28.239.194:38767)
I20260811 06:29:50.756397 2181 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.239.194@KRBTEST.COM'} at 127.28.239.194:40285
WARNING: no policy specified for kudu/127.28.239.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.239.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.239.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.195.keytab.
Entry for principal kudu/127.28.239.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.195.keytab.
I20260811 06:29:50.789455 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.195:0
--local_ip_for_outbound_sockets=127.28.239.195
--webserver_interface=127.28.239.195
--webserver_port=0
--tserver_master_addrs=127.28.239.254:41475
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.195.keytab
--principal=kudu/127.28.239.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:45937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:29:51.142496 2709 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:29:51.142827 2709 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:29:51.142931 2709 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:29:51.150486 2709 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:29:51.150631 2709 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:29:51.150756 2709 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.195
I20260811 06:29:51.161173 2709 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:45937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.195.keytab
--principal=kudu/127.28.239.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.239.195
--webserver_port=0
--tserver_master_addrs=127.28.239.254:41475
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.195
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:29:51.162618 2709 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:29:51.163992 2709 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:29:51.178866 2716 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:29:51.178838 2715 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:29:51.178915 2718 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:29:51.180503 2709 server_base.cc:1034] running on GCE node
I20260811 06:29:51.181221 2709 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:29:51.182377 2709 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:29:51.183616 2709 hybrid_clock.cc:648] HybridClock initialized: now 1786429791183543 us; error 68 us; skew 500 ppm
I20260811 06:29:51.184002 2709 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:29:51 dist-test-slave-gkw7 krb5kdc[2110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429791, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:29:51.188282 2709 init.cc:377] Logged in from keytab as kudu/127.28.239.195@KRBTEST.COM (short username kudu)
I20260811 06:29:51.190142 2709 webserver.cc:460] Webserver started at http://127.28.239.195:33807/ using document root <none> and password file <none>
I20260811 06:29:51.190912 2709 fs_manager.cc:359] Metadata directory not provided
I20260811 06:29:51.191057 2709 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:29:51.191433 2709 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:29:51.194796 2709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-2/data/instance:
uuid: "adcddf7ff20944ab8c20b449112d6739"
format_stamp: "Formatted at 2026-08-11 06:29:51 on dist-test-slave-gkw7"
server_key: "750fce658d2cecde7532dc9b06f42973"
server_key_iv: "7f839f0f35b1e85f8cbdfb46ae6827e2"
server_key_version: "encryptionkey@0"
I20260811 06:29:51.195668 2709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-2/wal/instance:
uuid: "adcddf7ff20944ab8c20b449112d6739"
format_stamp: "Formatted at 2026-08-11 06:29:51 on dist-test-slave-gkw7"
server_key: "750fce658d2cecde7532dc9b06f42973"
server_key_iv: "7f839f0f35b1e85f8cbdfb46ae6827e2"
server_key_version: "encryptionkey@0"
I20260811 06:29:51.201431 2709 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260811 06:29:51.205322 2725 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:29:51.206920 2709 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 06:29:51.207173 2709 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-2/wal
uuid: "adcddf7ff20944ab8c20b449112d6739"
format_stamp: "Formatted at 2026-08-11 06:29:51 on dist-test-slave-gkw7"
server_key: "750fce658d2cecde7532dc9b06f42973"
server_key_iv: "7f839f0f35b1e85f8cbdfb46ae6827e2"
server_key_version: "encryptionkey@0"
I20260811 06:29:51.207412 2709 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:29:51.223558 2481 heartbeater.cc:503] Master 127.28.239.254:41475 was elected leader, sending a full tablet report...
I20260811 06:29:51.248441 2709 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:29:51.253674 2709 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:29:51.253978 2709 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:29:51.255030 2709 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:29:51.256492 2709 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:29:51.256577 2709 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:29:51.256678 2709 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:29:51.256732 2709 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:29:51.301021 2709 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.195:41333
I20260811 06:29:51.301045 2914 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.195:41333 every 8 connection(s)
I20260811 06:29:51.302536 2709 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
I20260811 06:29:51.303704 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 2709
I20260811 06:29:51.303916 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786429755927508-29631-0/minicluster-data/ts-2/wal/instance
I20260811 06:29:51.304451 29631 external_mini_cluster.cc:1428] Setting key 5f25e44fa706c6f45f18f6b12cde0359
Aug 11 06:29:51 dist-test-slave-gkw7 krb5kdc[2110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429791, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.195@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:29:51.325001 2917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:51.305485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.195:38097 (local address 127.28.239.254:41475)
0811 06:29:51.306183 (+ 698us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:51.306191 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:51.307468 (+ 1277us) server_negotiation.cc:407] Connection header received
0811 06:29:51.309822 (+ 2354us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:51.309826 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:51.309907 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:51.309999 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:51.311632 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:51.312536 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:51.314432 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:51.314924 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:51.315056 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:51.318915 (+ 3859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:29:51.318920 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:29:51.318923 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:29:51.318958 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:29:51.321897 (+ 2939us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:51.322635 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:51.322645 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:51.322647 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:51.322773 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:51.323232 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:51.323237 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:51.323239 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:51.323516 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:29:51.323685 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:51.324391 (+ 706us) server_negotiation.cc:299] Negotiation successful
0811 06:29:51.324618 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":525,"thread_start_us":345,"threads_started":1}
I20260811 06:29:51.326092 2918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:51.306186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:41475 (local address 127.28.239.195:38097)
0811 06:29:51.307176 (+ 990us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:51.307234 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 06:29:51.309444 (+ 2210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:29:51.310258 (+ 814us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:29:51.310267 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:29:51.310847 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:29:51.311445 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:51.311465 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:51.312698 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:51.312707 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:51.314289 (+ 1582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:51.314303 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:51.315165 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:51.315172 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:51.315394 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:51.316219 (+ 825us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:29:51.316250 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:29:51.318669 (+ 2419us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:29:51.322042 (+ 3373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:29:51.322051 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:29:51.322068 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:29:51.322502 (+ 434us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:29:51.322894 (+ 392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:29:51.322899 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:29:51.322903 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:29:51.323105 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:29:51.323649 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:29:51.323657 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:29:51.324006 (+ 349us) client_negotiation.cc:769] Sending connection context
0811 06:29:51.324274 (+ 268us) client_negotiation.cc:240] Negotiation successful
0811 06:29:51.324827 (+ 553us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":656,"thread_start_us":371,"threads_started":1}
I20260811 06:29:51.327481 2915 heartbeater.cc:347] Connected to a master server at 127.28.239.254:41475
I20260811 06:29:51.327824 2915 heartbeater.cc:464] Registering TS with master...
I20260811 06:29:51.328594 2915 heartbeater.cc:511] Master 127.28.239.254:41475 requested a full tablet report, sending...
I20260811 06:29:51.330127 2181 ts_manager.cc:194] Registered new tserver with Master: adcddf7ff20944ab8c20b449112d6739 (127.28.239.195:41333)
I20260811 06:29:51.331323 2181 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.239.195@KRBTEST.COM'} at 127.28.239.195:38097
I20260811 06:29:51.333271 29631 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 06:29:51 dist-test-slave-gkw7 krb5kdc[2110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429791, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 11 06:29:51 dist-test-slave-gkw7 krb5kdc[2110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429791, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:29:51.389719 2917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:51.375039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36772 (local address 127.28.239.254:41475)
0811 06:29:51.375264 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:51.375288 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:51.375730 (+ 442us) server_negotiation.cc:407] Connection header received
0811 06:29:51.376045 (+ 315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:51.376050 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:51.376123 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:51.376215 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:51.377673 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:51.378524 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:51.379881 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:51.380289 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:51.380446 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:51.384439 (+ 3993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:29:51.384444 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:29:51.384447 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:29:51.384485 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:29:51.386987 (+ 2502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:51.387616 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:51.387623 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:51.387626 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:51.387757 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:51.388154 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:51.388160 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:51.388163 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:51.388521 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:29:51.388800 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:51.389108 (+ 308us) server_negotiation.cc:299] Negotiation successful
0811 06:29:51.389358 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
W20260811 06:29:51.390359 2181 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:36772
I20260811 06:29:51.392004 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 2277
Aug 11 06:29:51 dist-test-slave-gkw7 krb5kdc[2110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429791, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.28.239.193@KRBTEST.COM
I20260811 06:29:51.408449 2932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:51.393158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40170 (local address 127.28.239.193:43337)
0811 06:29:51.393801 (+ 643us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:51.393807 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:51.393833 (+ 26us) server_negotiation.cc:407] Connection header received
0811 06:29:51.393887 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:51.393893 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:51.394114 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:51.394301 (+ 187us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:51.395506 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:51.396571 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:51.397848 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:51.398437 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:51.398593 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:51.402610 (+ 4017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:29:51.402622 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:29:51.402637 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:29:51.402670 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:29:51.405630 (+ 2960us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:51.406290 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:51.406298 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:51.406300 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:51.406427 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:51.406823 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:51.406828 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:51.406833 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:51.407179 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:29:51.407510 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:51.407799 (+ 289us) server_negotiation.cc:299] Negotiation successful
0811 06:29:51.408078 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":495,"thread_start_us":304,"threads_started":1}
W20260811 06:29:51.409878 2474 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:40170
W20260811 06:29:51.411039 29631 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 2277: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 06:29:51.411228 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 2277
I20260811 06:29:51.438814 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 2494
Aug 11 06:29:51 dist-test-slave-gkw7 krb5kdc[2110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429791, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.28.239.194@KRBTEST.COM
I20260811 06:29:51.455344 2933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:51.439758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43502 (local address 127.28.239.194:38767)
0811 06:29:51.440424 (+ 666us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:51.440431 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:51.440458 (+ 27us) server_negotiation.cc:407] Connection header received
0811 06:29:51.440526 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:51.440532 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:51.440763 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:51.440905 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:51.442292 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:51.443483 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:51.444737 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:51.445342 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:51.445448 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:51.449756 (+ 4308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:29:51.449770 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:29:51.449786 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:29:51.449828 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:29:51.452846 (+ 3018us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:51.453457 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:51.453464 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:51.453466 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:51.453654 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:51.453978 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:51.453982 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:51.453984 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:51.454325 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:29:51.454562 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:51.454830 (+ 268us) server_negotiation.cc:299] Negotiation successful
0811 06:29:51.455056 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":505,"thread_start_us":363,"threads_started":1}
W20260811 06:29:51.456686 2688 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:43502
W20260811 06:29:51.457906 29631 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 2494: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 06:29:51.458094 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 2494
I20260811 06:29:51.484165 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 2709
Aug 11 06:29:51 dist-test-slave-gkw7 krb5kdc[2110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429791, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.28.239.195@KRBTEST.COM
I20260811 06:29:51.500963 2934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:51.485123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44952 (local address 127.28.239.195:41333)
0811 06:29:51.485831 (+ 708us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:51.485840 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:51.485872 (+ 32us) server_negotiation.cc:407] Connection header received
0811 06:29:51.485936 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:51.485943 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:51.486161 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:51.486303 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:51.487604 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:51.488662 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:51.490058 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:51.490634 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:51.490761 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:51.495226 (+ 4465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:29:51.495248 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:29:51.495264 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:29:51.495301 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:29:51.498382 (+ 3081us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:51.499003 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:51.499010 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:51.499013 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:51.499145 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:51.499470 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:51.499473 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:51.499476 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:51.499827 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:29:51.500078 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:51.500331 (+ 253us) server_negotiation.cc:299] Negotiation successful
0811 06:29:51.500534 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":543,"thread_start_us":336,"threads_started":1}
W20260811 06:29:51.502445 2908 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:44952
W20260811 06:29:51.503599 29631 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 2709: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 06:29:51.503804 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 2709
I20260811 06:29:51.528170 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 2129
I20260811 06:29:51.529498 2224 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:29:51.636981 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 2129
2026-08-11T06:29:51Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (4195 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:29:51 dist-test-slave-gkw7 krb5kdc[2942](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:29:51 dist-test-slave-gkw7 krb5kdc[2942](info): set up 2 sockets
Aug 11 06:29:51 dist-test-slave-gkw7 krb5kdc[2942](info): commencing operation
krb5kdc: starting...
W20260811 06:29:52.966068 29631 mini_kdc.cc:121] Time spent starting KDC: real 1.275s user 0.002s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/krb5kdc/test-user.keytab.
Aug 11 06:29:53 dist-test-slave-gkw7 krb5kdc[2942](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429793, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:29:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:29:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.239.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.239.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.239.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab.
Entry for principal kudu/127.28.239.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab.
I20260811 06:29:53.198208 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:39457
--webserver_interface=127.28.239.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab
--principal=kudu/127.28.239.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:39709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.239.254:39457
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:29:53.550729 2959 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:29:53.551041 2959 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:29:53.551131 2959 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:29:53.558343 2959 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:29:53.558463 2959 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:29:53.558512 2959 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:29:53.558564 2959 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:29:53.558611 2959 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:29:53.568707 2959 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:39709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.239.254:39457
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab
--principal=kudu/127.28.239.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:39457
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.239.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:29:53.570264 2959 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:29:53.571625 2959 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:29:53.584380 2966 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:29:53.584429 2965 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:29:53.585064 2968 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:29:53.585129 2959 server_base.cc:1034] running on GCE node
I20260811 06:29:53.586072 2959 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:29:53.587610 2959 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:29:53.588832 2959 hybrid_clock.cc:648] HybridClock initialized: now 1786429793588774 us; error 50 us; skew 500 ppm
I20260811 06:29:53.589224 2959 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:29:53 dist-test-slave-gkw7 krb5kdc[2942](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429793, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:29:53.592624 2959 init.cc:377] Logged in from keytab as kudu/127.28.239.254@KRBTEST.COM (short username kudu)
I20260811 06:29:53.594251 2959 webserver.cc:460] Webserver started at http://127.28.239.254:40135/ using document root <none> and password file <none>
I20260811 06:29:53.594935 2959 fs_manager.cc:359] Metadata directory not provided
I20260811 06:29:53.595036 2959 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:29:53.595311 2959 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:29:53.598431 2959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/minicluster-data/master-0/data/instance:
uuid: "04b54444fff24189a63207adff11e2a5"
format_stamp: "Formatted at 2026-08-11 06:29:53 on dist-test-slave-gkw7"
server_key: "c9d3a51fc8f82352609385cec8f26762"
server_key_iv: "dc9d2edf02d62a047a47c69765d1b1eb"
server_key_version: "encryptionkey@0"
I20260811 06:29:53.599206 2959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/minicluster-data/master-0/wal/instance:
uuid: "04b54444fff24189a63207adff11e2a5"
format_stamp: "Formatted at 2026-08-11 06:29:53 on dist-test-slave-gkw7"
server_key: "c9d3a51fc8f82352609385cec8f26762"
server_key_iv: "dc9d2edf02d62a047a47c69765d1b1eb"
server_key_version: "encryptionkey@0"
I20260811 06:29:53.604578 2959 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 06:29:53.608206 2983 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:29:53.609839 2959 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:29:53.610049 2959 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/minicluster-data/master-0/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/minicluster-data/master-0/wal
uuid: "04b54444fff24189a63207adff11e2a5"
format_stamp: "Formatted at 2026-08-11 06:29:53 on dist-test-slave-gkw7"
server_key: "c9d3a51fc8f82352609385cec8f26762"
server_key_iv: "dc9d2edf02d62a047a47c69765d1b1eb"
server_key_version: "encryptionkey@0"
I20260811 06:29:53.610241 2959 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:29:53.637511 2959 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:29:53.642951 2959 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:29:53.643307 2959 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:29:53.664114 2959 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.254:39457
I20260811 06:29:53.664127 3055 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.254:39457 every 8 connection(s)
I20260811 06:29:53.665743 2959 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
I20260811 06:29:53.669641 3056 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:29:53.671896 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 2959
I20260811 06:29:53.672200 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/minicluster-data/master-0/wal/instance
I20260811 06:29:53.672931 29631 external_mini_cluster.cc:1428] Setting key e3f98f35e2d209784ab9afe4e2d84d48
I20260811 06:29:53.683077 3056 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04b54444fff24189a63207adff11e2a5: Bootstrap starting.
I20260811 06:29:53.686734 3056 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 04b54444fff24189a63207adff11e2a5: Neither blocks nor log segments found. Creating new log.
I20260811 06:29:53.687796 3056 log.cc:824] T 00000000000000000000000000000000 P 04b54444fff24189a63207adff11e2a5: Log is configured to *not* fsync() on all Append() calls
Aug 11 06:29:53 dist-test-slave-gkw7 krb5kdc[2942](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429793, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:29:53.691332 3056 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04b54444fff24189a63207adff11e2a5: No bootstrap required, opened a new log
I20260811 06:29:53.700196 3060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:53.675368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39388 (local address 127.28.239.254:39457)
0811 06:29:53.676166 (+ 798us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:53.676181 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:53.676231 (+ 50us) server_negotiation.cc:407] Connection header received
0811 06:29:53.678621 (+ 2390us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:53.678654 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:53.679198 (+ 544us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:53.679711 (+ 513us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:53.681239 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:53.683324 (+ 2085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:53.684809 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:53.685916 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:53.686104 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:53.690979 (+ 4875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:29:53.690991 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:29:53.691010 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:29:53.691052 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:29:53.695223 (+ 4171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:53.695969 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:53.695979 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:53.695987 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:53.696214 (+ 227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:53.696611 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:53.696615 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:53.696618 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:53.697377 (+ 759us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:29:53.697727 (+ 350us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:53.698181 (+ 454us) server_negotiation.cc:299] Negotiation successful
0811 06:29:53.698652 (+ 471us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":556,"thread_start_us":330,"threads_started":1}
I20260811 06:29:53.702591 3056 raft_consensus.cc:348] T 00000000000000000000000000000000 P 04b54444fff24189a63207adff11e2a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04b54444fff24189a63207adff11e2a5" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 39457 } }
I20260811 06:29:53.702998 3056 raft_consensus.cc:374] T 00000000000000000000000000000000 P 04b54444fff24189a63207adff11e2a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:29:53.703131 3056 raft_consensus.cc:729] T 00000000000000000000000000000000 P 04b54444fff24189a63207adff11e2a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04b54444fff24189a63207adff11e2a5, State: Initialized, Role: FOLLOWER
I20260811 06:29:53.703796 3056 consensus_queue.cc:260] T 00000000000000000000000000000000 P 04b54444fff24189a63207adff11e2a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04b54444fff24189a63207adff11e2a5" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 39457 } }
I20260811 06:29:53.704051 3056 raft_consensus.cc:388] T 00000000000000000000000000000000 P 04b54444fff24189a63207adff11e2a5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:29:53.704187 3056 raft_consensus.cc:482] T 00000000000000000000000000000000 P 04b54444fff24189a63207adff11e2a5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:29:53.704382 3056 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 04b54444fff24189a63207adff11e2a5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:29:53.706703 3056 raft_consensus.cc:504] T 00000000000000000000000000000000 P 04b54444fff24189a63207adff11e2a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04b54444fff24189a63207adff11e2a5" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 39457 } }
I20260811 06:29:53.707224 3056 leader_election.cc:302] T 00000000000000000000000000000000 P 04b54444fff24189a63207adff11e2a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 04b54444fff24189a63207adff11e2a5; no voters:
I20260811 06:29:53.707870 3056 leader_election.cc:288] T 00000000000000000000000000000000 P 04b54444fff24189a63207adff11e2a5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:29:53.708058 3062 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 04b54444fff24189a63207adff11e2a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:29:53.709153 3062 raft_consensus.cc:686] T 00000000000000000000000000000000 P 04b54444fff24189a63207adff11e2a5 [term 1 LEADER]: Becoming Leader. State: Replica: 04b54444fff24189a63207adff11e2a5, State: Running, Role: LEADER
I20260811 06:29:53.709780 3062 consensus_queue.cc:237] T 00000000000000000000000000000000 P 04b54444fff24189a63207adff11e2a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04b54444fff24189a63207adff11e2a5" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 39457 } }
I20260811 06:29:53.710292 3056 sys_catalog.cc:564] T 00000000000000000000000000000000 P 04b54444fff24189a63207adff11e2a5 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:29:53.712536 3065 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04b54444fff24189a63207adff11e2a5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 04b54444fff24189a63207adff11e2a5. Latest consensus state: current_term: 1 leader_uuid: "04b54444fff24189a63207adff11e2a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04b54444fff24189a63207adff11e2a5" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 39457 } } }
I20260811 06:29:53.712965 3065 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04b54444fff24189a63207adff11e2a5 [sys.catalog]: This master's current role is: LEADER
I20260811 06:29:53.714437 3064 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04b54444fff24189a63207adff11e2a5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "04b54444fff24189a63207adff11e2a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04b54444fff24189a63207adff11e2a5" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 39457 } } }
I20260811 06:29:53.714926 3064 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04b54444fff24189a63207adff11e2a5 [sys.catalog]: This master's current role is: LEADER
I20260811 06:29:53.716571 3075 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:29:53.721024 3075 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:29:53.730119 3075 catalog_manager.cc:1347] Generated new cluster ID: 9ad5d1e355294d2d93f96ef49d531e09
I20260811 06:29:53.730244 3075 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:29:53.754354 3075 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:29:53.755846 3075 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:29:53.765926 3075 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 04b54444fff24189a63207adff11e2a5: Generated new TSK 0
I20260811 06:29:53.766984 3075 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.239.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.239.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.239.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.193.keytab.
Entry for principal kudu/127.28.239.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.193.keytab.
I20260811 06:29:53.825383 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.193:0
--local_ip_for_outbound_sockets=127.28.239.193
--webserver_interface=127.28.239.193
--webserver_port=0
--tserver_master_addrs=127.28.239.254:39457
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.193.keytab
--principal=kudu/127.28.239.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:39709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:29:54.175758 3096 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:29:54.176054 3096 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:29:54.176201 3096 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:29:54.183941 3096 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:29:54.184056 3096 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:29:54.184258 3096 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.193
I20260811 06:29:54.194464 3096 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:39709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.193.keytab
--principal=kudu/127.28.239.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.239.193
--webserver_port=0
--tserver_master_addrs=127.28.239.254:39457
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.193
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:29:54.195813 3096 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:29:54.197227 3096 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:29:54.211213 3110 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:29:54.211294 3113 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:29:54.211400 3111 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:29:54.212754 3096 server_base.cc:1034] running on GCE node
I20260811 06:29:54.213442 3096 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:29:54.214740 3096 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:29:54.215986 3096 hybrid_clock.cc:648] HybridClock initialized: now 1786429794215914 us; error 60 us; skew 500 ppm
I20260811 06:29:54.216333 3096 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:29:54 dist-test-slave-gkw7 krb5kdc[2942](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429794, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:29:54.219914 3096 init.cc:377] Logged in from keytab as kudu/127.28.239.193@KRBTEST.COM (short username kudu)
I20260811 06:29:54.221434 3096 webserver.cc:460] Webserver started at http://127.28.239.193:37905/ using document root <none> and password file <none>
I20260811 06:29:54.222200 3096 fs_manager.cc:359] Metadata directory not provided
I20260811 06:29:54.222306 3096 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:29:54.222581 3096 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:29:54.225509 3096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/minicluster-data/ts-0/data/instance:
uuid: "352e4aad9a4d49229dd7e44aee313b10"
format_stamp: "Formatted at 2026-08-11 06:29:54 on dist-test-slave-gkw7"
server_key: "7cc7ca058e2cec18a33138fad3ae79f6"
server_key_iv: "75ead003d2c93577a67fce58ff6b4ea0"
server_key_version: "encryptionkey@0"
I20260811 06:29:54.226361 3096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/minicluster-data/ts-0/wal/instance:
uuid: "352e4aad9a4d49229dd7e44aee313b10"
format_stamp: "Formatted at 2026-08-11 06:29:54 on dist-test-slave-gkw7"
server_key: "7cc7ca058e2cec18a33138fad3ae79f6"
server_key_iv: "75ead003d2c93577a67fce58ff6b4ea0"
server_key_version: "encryptionkey@0"
I20260811 06:29:54.231626 3096 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 06:29:54.235483 3127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:29:54.236898 3096 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 06:29:54.237116 3096 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/minicluster-data/ts-0/wal
uuid: "352e4aad9a4d49229dd7e44aee313b10"
format_stamp: "Formatted at 2026-08-11 06:29:54 on dist-test-slave-gkw7"
server_key: "7cc7ca058e2cec18a33138fad3ae79f6"
server_key_iv: "75ead003d2c93577a67fce58ff6b4ea0"
server_key_version: "encryptionkey@0"
I20260811 06:29:54.237360 3096 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:29:54.254866 3096 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:29:54.259678 3096 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:29:54.260010 3096 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:29:54.261077 3096 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:29:54.262586 3096 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:29:54.262706 3096 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:29:54.262861 3096 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:29:54.262935 3096 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:29:54.298782 3096 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.193:39973
I20260811 06:29:54.298831 3304 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.193:39973 every 8 connection(s)
I20260811 06:29:54.300210 3096 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
I20260811 06:29:54.310416 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 3096
I20260811 06:29:54.310676 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/minicluster-data/ts-0/wal/instance
I20260811 06:29:54.311522 29631 external_mini_cluster.cc:1428] Setting key 56ede02fa406c632891b12d0f98453dc
Aug 11 06:29:54 dist-test-slave-gkw7 krb5kdc[2942](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429794, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.193@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:29:54.323024 3313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:54.303095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.193:51939 (local address 127.28.239.254:39457)
0811 06:29:54.303576 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:54.303582 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:54.304667 (+ 1085us) server_negotiation.cc:407] Connection header received
0811 06:29:54.306631 (+ 1964us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:54.306636 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:54.306712 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:54.306833 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:54.308343 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:54.309093 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:54.310994 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:54.311730 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:54.311879 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:54.316558 (+ 4679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:29:54.316563 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:29:54.316566 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:29:54.316598 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:29:54.319416 (+ 2818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:54.320488 (+ 1072us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:54.320491 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:54.320494 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:54.320611 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:54.321146 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:54.321149 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:54.321151 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:54.321413 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:29:54.321571 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:54.322582 (+ 1011us) server_negotiation.cc:299] Negotiation successful
0811 06:29:54.322757 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":221,"threads_started":1}
I20260811 06:29:54.324352 3317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:54.303750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:39457 (local address 127.28.239.193:51939)
0811 06:29:54.304495 (+ 745us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:54.304540 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 06:29:54.306333 (+ 1793us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:29:54.307049 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:29:54.307057 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:29:54.307568 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:29:54.308143 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:54.308162 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:54.309254 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:54.309258 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:54.310844 (+ 1586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:54.310857 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:54.311973 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:54.311978 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:54.312206 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:54.313057 (+ 851us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:29:54.313097 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:29:54.316331 (+ 3234us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:29:54.319583 (+ 3252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:29:54.319596 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:29:54.319618 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:29:54.320353 (+ 735us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:29:54.320731 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:29:54.320739 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:29:54.320746 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:29:54.321024 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:29:54.321578 (+ 554us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:29:54.321590 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:29:54.322127 (+ 537us) client_negotiation.cc:769] Sending connection context
0811 06:29:54.322492 (+ 365us) client_negotiation.cc:240] Negotiation successful
0811 06:29:54.323149 (+ 657us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":301,"threads_started":1}
I20260811 06:29:54.326258 3309 heartbeater.cc:347] Connected to a master server at 127.28.239.254:39457
I20260811 06:29:54.326613 3309 heartbeater.cc:464] Registering TS with master...
I20260811 06:29:54.327430 3309 heartbeater.cc:511] Master 127.28.239.254:39457 requested a full tablet report, sending...
I20260811 06:29:54.329856 3006 ts_manager.cc:194] Registered new tserver with Master: 352e4aad9a4d49229dd7e44aee313b10 (127.28.239.193:39973)
I20260811 06:29:54.331775 3006 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.239.193@KRBTEST.COM'} at 127.28.239.193:51939
WARNING: no policy specified for kudu/127.28.239.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.239.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.239.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.194.keytab.
Entry for principal kudu/127.28.239.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.194.keytab.
I20260811 06:29:54.366840 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.194:0
--local_ip_for_outbound_sockets=127.28.239.194
--webserver_interface=127.28.239.194
--webserver_port=0
--tserver_master_addrs=127.28.239.254:39457
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.194.keytab
--principal=kudu/127.28.239.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:39709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:29:54.730413 3321 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:29:54.730767 3321 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:29:54.730983 3321 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:29:54.738862 3321 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:29:54.739059 3321 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:29:54.739252 3321 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.194
I20260811 06:29:54.750015 3321 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:39709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.194.keytab
--principal=kudu/127.28.239.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.239.194
--webserver_port=0
--tserver_master_addrs=127.28.239.254:39457
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.194
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:29:54.751499 3321 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:29:54.752943 3321 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:29:54.766765 3333 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:29:54.766819 3335 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:29:54.766855 3337 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:29:54.768136 3321 server_base.cc:1034] running on GCE node
I20260811 06:29:54.768697 3321 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:29:54.769690 3321 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:29:54.770931 3321 hybrid_clock.cc:648] HybridClock initialized: now 1786429794770879 us; error 58 us; skew 500 ppm
I20260811 06:29:54.771299 3321 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:29:54 dist-test-slave-gkw7 krb5kdc[2942](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429794, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:29:54.778689 3321 init.cc:377] Logged in from keytab as kudu/127.28.239.194@KRBTEST.COM (short username kudu)
I20260811 06:29:54.780323 3321 webserver.cc:460] Webserver started at http://127.28.239.194:39729/ using document root <none> and password file <none>
I20260811 06:29:54.781057 3321 fs_manager.cc:359] Metadata directory not provided
I20260811 06:29:54.781211 3321 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:29:54.781610 3321 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:29:54.784683 3321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/minicluster-data/ts-1/data/instance:
uuid: "23e06bb420704f0e863f0a3e59ad570b"
format_stamp: "Formatted at 2026-08-11 06:29:54 on dist-test-slave-gkw7"
server_key: "129b3ebcbed9ba656052366b9e96d0bd"
server_key_iv: "cebe3f796d49e2ecc1a2eb17eee2b7b7"
server_key_version: "encryptionkey@0"
I20260811 06:29:54.785636 3321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/minicluster-data/ts-1/wal/instance:
uuid: "23e06bb420704f0e863f0a3e59ad570b"
format_stamp: "Formatted at 2026-08-11 06:29:54 on dist-test-slave-gkw7"
server_key: "129b3ebcbed9ba656052366b9e96d0bd"
server_key_iv: "cebe3f796d49e2ecc1a2eb17eee2b7b7"
server_key_version: "encryptionkey@0"
I20260811 06:29:54.791025 3321 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 06:29:54.794756 3350 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:29:54.796140 3321 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 06:29:54.796331 3321 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/minicluster-data/ts-1/wal
uuid: "23e06bb420704f0e863f0a3e59ad570b"
format_stamp: "Formatted at 2026-08-11 06:29:54 on dist-test-slave-gkw7"
server_key: "129b3ebcbed9ba656052366b9e96d0bd"
server_key_iv: "cebe3f796d49e2ecc1a2eb17eee2b7b7"
server_key_version: "encryptionkey@0"
I20260811 06:29:54.796547 3321 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:29:54.822801 3321 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:29:54.827206 3321 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:29:54.827534 3321 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:29:54.828584 3321 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:29:54.830084 3321 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:29:54.830194 3321 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:29:54.830363 3321 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:29:54.830441 3321 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:29:54.865857 3321 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.194:43611
I20260811 06:29:54.865916 3536 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.194:43611 every 8 connection(s)
I20260811 06:29:54.867261 3321 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
I20260811 06:29:54.873003 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 3321
I20260811 06:29:54.873215 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/minicluster-data/ts-1/wal/instance
I20260811 06:29:54.873816 29631 external_mini_cluster.cc:1428] Setting key 38b1149694f3904f4a781c41b4bcfa97
Aug 11 06:29:54 dist-test-slave-gkw7 krb5kdc[2942](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429794, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.194@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:29:54.890012 3540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:54.870013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.194:35211 (local address 127.28.239.254:39457)
0811 06:29:54.870489 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:54.870496 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:54.871527 (+ 1031us) server_negotiation.cc:407] Connection header received
0811 06:29:54.873810 (+ 2283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:54.873815 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:54.873888 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:54.874029 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:54.875926 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:54.876671 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:54.879322 (+ 2651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:54.879737 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:54.879847 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:54.884110 (+ 4263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:29:54.884115 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:29:54.884118 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:29:54.884153 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:29:54.886808 (+ 2655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:54.887583 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:54.887587 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:54.887589 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:54.887715 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:54.888184 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:54.888188 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:54.888191 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:54.888459 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:29:54.888622 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:54.889351 (+ 729us) server_negotiation.cc:299] Negotiation successful
0811 06:29:54.889610 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":214,"threads_started":1}
I20260811 06:29:54.891121 3541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:54.870636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:39457 (local address 127.28.239.194:35211)
0811 06:29:54.871366 (+ 730us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:54.871405 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 06:29:54.873478 (+ 2073us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:29:54.874319 (+ 841us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:29:54.874331 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:29:54.875046 (+ 715us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:29:54.875747 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:54.875770 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:54.876842 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:54.876850 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:54.879138 (+ 2288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:54.879155 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:54.879976 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:54.879985 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:54.880320 (+ 335us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:54.881070 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:29:54.881107 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:29:54.883863 (+ 2756us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:29:54.886964 (+ 3101us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:29:54.886973 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:29:54.886990 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:29:54.887440 (+ 450us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:29:54.887841 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:29:54.887846 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:29:54.887850 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:29:54.888049 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:29:54.888601 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:29:54.888609 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:29:54.888956 (+ 347us) client_negotiation.cc:769] Sending connection context
0811 06:29:54.889237 (+ 281us) client_negotiation.cc:240] Negotiation successful
0811 06:29:54.889846 (+ 609us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":283,"threads_started":1}
I20260811 06:29:54.892750 3537 heartbeater.cc:347] Connected to a master server at 127.28.239.254:39457
I20260811 06:29:54.893075 3537 heartbeater.cc:464] Registering TS with master...
I20260811 06:29:54.893896 3537 heartbeater.cc:511] Master 127.28.239.254:39457 requested a full tablet report, sending...
I20260811 06:29:54.895450 3006 ts_manager.cc:194] Registered new tserver with Master: 23e06bb420704f0e863f0a3e59ad570b (127.28.239.194:43611)
WARNING: no policy specified for kudu/127.28.239.195@KRBTEST.COM; defaulting to no policy
I20260811 06:29:54.896287 3006 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.239.194@KRBTEST.COM'} at 127.28.239.194:35211
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.239.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.239.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.195.keytab.
Entry for principal kudu/127.28.239.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.195.keytab.
I20260811 06:29:54.924753 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.195:0
--local_ip_for_outbound_sockets=127.28.239.195
--webserver_interface=127.28.239.195
--webserver_port=0
--tserver_master_addrs=127.28.239.254:39457
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.195.keytab
--principal=kudu/127.28.239.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:39709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:29:55.267743 3546 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:29:55.268085 3546 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:29:55.268265 3546 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:29:55.278088 3546 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:29:55.278298 3546 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:29:55.278489 3546 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.195
I20260811 06:29:55.289000 3546 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:39709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.195.keytab
--principal=kudu/127.28.239.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.239.195
--webserver_port=0
--tserver_master_addrs=127.28.239.254:39457
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.195
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:29:55.290416 3546 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:29:55.291782 3546 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:29:55.307755 3559 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:29:55.307797 3560 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:29:55.308058 3565 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:29:55.308899 3546 server_base.cc:1034] running on GCE node
I20260811 06:29:55.309660 3546 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:29:55.310674 3546 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:29:55.311916 3546 hybrid_clock.cc:648] HybridClock initialized: now 1786429795311871 us; error 45 us; skew 500 ppm
I20260811 06:29:55.312263 3546 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:29:55 dist-test-slave-gkw7 krb5kdc[2942](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429795, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:29:55.315991 3546 init.cc:377] Logged in from keytab as kudu/127.28.239.195@KRBTEST.COM (short username kudu)
I20260811 06:29:55.317499 3546 webserver.cc:460] Webserver started at http://127.28.239.195:35791/ using document root <none> and password file <none>
I20260811 06:29:55.318295 3546 fs_manager.cc:359] Metadata directory not provided
I20260811 06:29:55.318450 3546 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:29:55.318778 3546 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:29:55.321908 3546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/minicluster-data/ts-2/data/instance:
uuid: "17fdc706e6c24d43beb1fb392e755cec"
format_stamp: "Formatted at 2026-08-11 06:29:55 on dist-test-slave-gkw7"
server_key: "03674f802a27d1329a64db1313a37f75"
server_key_iv: "85738073937a83ede6215f7ffc3138d0"
server_key_version: "encryptionkey@0"
I20260811 06:29:55.322745 3546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/minicluster-data/ts-2/wal/instance:
uuid: "17fdc706e6c24d43beb1fb392e755cec"
format_stamp: "Formatted at 2026-08-11 06:29:55 on dist-test-slave-gkw7"
server_key: "03674f802a27d1329a64db1313a37f75"
server_key_iv: "85738073937a83ede6215f7ffc3138d0"
server_key_version: "encryptionkey@0"
I20260811 06:29:55.328116 3546 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 06:29:55.331872 3575 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:29:55.333304 3546 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 06:29:55.333499 3546 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/minicluster-data/ts-2/wal
uuid: "17fdc706e6c24d43beb1fb392e755cec"
format_stamp: "Formatted at 2026-08-11 06:29:55 on dist-test-slave-gkw7"
server_key: "03674f802a27d1329a64db1313a37f75"
server_key_iv: "85738073937a83ede6215f7ffc3138d0"
server_key_version: "encryptionkey@0"
I20260811 06:29:55.333774 3546 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:29:55.335160 3309 heartbeater.cc:503] Master 127.28.239.254:39457 was elected leader, sending a full tablet report...
I20260811 06:29:55.352958 3546 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:29:55.357669 3546 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:29:55.357982 3546 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:29:55.359012 3546 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:29:55.360422 3546 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:29:55.360530 3546 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:29:55.360718 3546 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:29:55.360795 3546 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:29:55.397228 3546 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.195:36985
I20260811 06:29:55.397292 3750 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.195:36985 every 8 connection(s)
I20260811 06:29:55.398654 3546 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
I20260811 06:29:55.408077 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 3546
I20260811 06:29:55.408291 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786429755927508-29631-0/minicluster-data/ts-2/wal/instance
I20260811 06:29:55.408757 29631 external_mini_cluster.cc:1428] Setting key 294d65aa000dfb18b04ef1393989555f
Aug 11 06:29:55 dist-test-slave-gkw7 krb5kdc[2942](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429795, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.195@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:29:55.420295 3753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:55.401479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.195:47733 (local address 127.28.239.254:39457)
0811 06:29:55.402003 (+ 524us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:55.402008 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:55.403019 (+ 1011us) server_negotiation.cc:407] Connection header received
0811 06:29:55.404895 (+ 1876us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:55.404901 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:55.405008 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:55.405132 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:55.406771 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:55.407773 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:55.409657 (+ 1884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:55.410319 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:55.410467 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:55.414395 (+ 3928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:29:55.414400 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:29:55.414404 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:29:55.414437 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:29:55.417006 (+ 2569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:55.417829 (+ 823us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:55.417835 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:55.417838 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:55.417972 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:55.418433 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:55.418438 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:55.418442 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:55.418737 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:29:55.418916 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:55.419641 (+ 725us) server_negotiation.cc:299] Negotiation successful
0811 06:29:55.419874 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":244,"threads_started":1}
I20260811 06:29:55.421267 3755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:55.402124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:39457 (local address 127.28.239.195:47733)
0811 06:29:55.402855 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:55.402892 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 06:29:55.404616 (+ 1724us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:29:55.405398 (+ 782us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:29:55.405406 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:29:55.405991 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:29:55.406589 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:55.406606 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:55.407934 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:55.407938 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:55.409448 (+ 1510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:55.409467 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:55.410562 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:55.410572 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:55.410890 (+ 318us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:55.411742 (+ 852us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:29:55.411776 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:29:55.414118 (+ 2342us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:29:55.417149 (+ 3031us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:29:55.417158 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:29:55.417175 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:29:55.417690 (+ 515us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:29:55.418090 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:29:55.418095 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:29:55.418099 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:29:55.418303 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:29:55.418880 (+ 577us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:29:55.418888 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:29:55.419247 (+ 359us) client_negotiation.cc:769] Sending connection context
0811 06:29:55.419530 (+ 283us) client_negotiation.cc:240] Negotiation successful
0811 06:29:55.420102 (+ 572us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":285,"threads_started":1}
I20260811 06:29:55.422974 3751 heartbeater.cc:347] Connected to a master server at 127.28.239.254:39457
I20260811 06:29:55.423342 3751 heartbeater.cc:464] Registering TS with master...
I20260811 06:29:55.424290 3751 heartbeater.cc:511] Master 127.28.239.254:39457 requested a full tablet report, sending...
I20260811 06:29:55.425791 3006 ts_manager.cc:194] Registered new tserver with Master: 17fdc706e6c24d43beb1fb392e755cec (127.28.239.195:36985)
I20260811 06:29:55.426615 3006 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.239.195@KRBTEST.COM'} at 127.28.239.195:47733
I20260811 06:29:55.437039 29631 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 06:29:55 dist-test-slave-gkw7 krb5kdc[2942](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429795, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 06:29:55 dist-test-slave-gkw7 krb5kdc[2942](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429795, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.239.193@KRBTEST.COM
I20260811 06:29:55.496886 3762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:55.479476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44832 (local address 127.28.239.193:39973)
0811 06:29:55.480135 (+ 659us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:55.480143 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:55.480172 (+ 29us) server_negotiation.cc:407] Connection header received
0811 06:29:55.480257 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:55.480264 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:55.480499 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:55.480633 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:55.482355 (+ 1722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:55.483384 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:55.484729 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:55.485324 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:55.485452 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:55.490159 (+ 4707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:29:55.490174 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:29:55.490191 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:29:55.490232 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:29:55.493619 (+ 3387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:55.494427 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:55.494447 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:55.494450 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:55.494592 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:55.495019 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:55.495025 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:55.495031 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:55.495441 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:29:55.495801 (+ 360us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:55.496165 (+ 364us) server_negotiation.cc:299] Negotiation successful
0811 06:29:55.496479 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":492,"thread_start_us":346,"threads_started":1}
Aug 11 06:29:55 dist-test-slave-gkw7 krb5kdc[2942](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429795, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:29:55.516575 3753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:55.502432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39400 (local address 127.28.239.254:39457)
0811 06:29:55.502683 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:55.502687 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:55.502797 (+ 110us) server_negotiation.cc:407] Connection header received
0811 06:29:55.502928 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:55.502932 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:55.503013 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:55.503097 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:55.504381 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:55.505125 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:55.506413 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:55.506821 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:55.506918 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:55.510963 (+ 4045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:29:55.510968 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:29:55.510971 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:29:55.511006 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:29:55.513928 (+ 2922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:55.514675 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:55.514679 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:55.514681 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:55.514822 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:55.515174 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:55.515177 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:55.515179 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:55.515567 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:29:55.515763 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:55.516039 (+ 276us) server_negotiation.cc:299] Negotiation successful
0811 06:29:55.516213 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
W20260811 06:29:55.517163 3006 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39400
I20260811 06:29:55.518244 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 3096
I20260811 06:29:55.531209 3762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:55.519230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44848 (local address 127.28.239.193:39973)
0811 06:29:55.519448 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:55.519455 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:55.519618 (+ 163us) server_negotiation.cc:407] Connection header received
0811 06:29:55.519831 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:55.519836 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:55.519902 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:55.520002 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:55.521219 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:55.522011 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:55.523268 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:55.523706 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:55.523847 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:55.526139 (+ 2292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:29:55.526143 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:29:55.526147 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:29:55.526181 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:29:55.528636 (+ 2455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:55.529270 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:55.529275 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:55.529279 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:55.529413 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:55.529826 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:55.529832 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:55.529836 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:55.530092 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:29:55.530356 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:55.530652 (+ 296us) server_negotiation.cc:299] Negotiation successful
0811 06:29:55.530881 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 06:29:55.531956 3300 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:29:55.617947 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 3096
I20260811 06:29:55.642933 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 3321
Aug 11 06:29:55 dist-test-slave-gkw7 krb5kdc[2942](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429795, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.239.194@KRBTEST.COM
I20260811 06:29:55.659272 3767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:55.643843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51410 (local address 127.28.239.194:43611)
0811 06:29:55.644387 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:55.644395 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:55.644422 (+ 27us) server_negotiation.cc:407] Connection header received
0811 06:29:55.644487 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:55.644493 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:55.644732 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:55.644869 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:55.646227 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:55.647246 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:55.648500 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:55.649091 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:55.649227 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:55.653639 (+ 4412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:29:55.653653 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:29:55.653671 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:29:55.653713 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:29:55.656703 (+ 2990us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:55.657308 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:55.657315 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:55.657317 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:55.657449 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:55.657819 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:55.657823 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:55.657825 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:55.658167 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:29:55.658406 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:55.658672 (+ 266us) server_negotiation.cc:299] Negotiation successful
0811 06:29:55.658937 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":268,"threads_started":1}
I20260811 06:29:55.660523 3532 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:29:55.750448 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 3321
I20260811 06:29:55.777118 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 3546
Aug 11 06:29:55 dist-test-slave-gkw7 krb5kdc[2942](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429795, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.239.195@KRBTEST.COM
I20260811 06:29:55.793740 3770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:55.778141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48800 (local address 127.28.239.195:36985)
0811 06:29:55.778735 (+ 594us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:55.778742 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:55.778771 (+ 29us) server_negotiation.cc:407] Connection header received
0811 06:29:55.778831 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:55.778837 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:55.779049 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:55.779180 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:55.780465 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:55.781515 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:55.782960 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:55.783586 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:55.783688 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:55.787910 (+ 4222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:29:55.787921 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:29:55.787936 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:29:55.787967 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:29:55.791068 (+ 3101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:55.791811 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:55.791818 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:55.791820 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:55.791941 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:55.792255 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:55.792258 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:55.792260 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:55.792597 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:29:55.792831 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:55.793071 (+ 240us) server_negotiation.cc:299] Negotiation successful
0811 06:29:55.793284 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":443,"thread_start_us":276,"threads_started":1}
I20260811 06:29:55.795081 3744 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:29:55.882665 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 3546
I20260811 06:29:55.909670 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 2959
I20260811 06:29:55.910980 3050 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:29:56.017660 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 2959
2026-08-11T06:29:56Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (4378 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:29:56 dist-test-slave-gkw7 krb5kdc[3779](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:29:56 dist-test-slave-gkw7 krb5kdc[3779](info): set up 2 sockets
Aug 11 06:29:56 dist-test-slave-gkw7 krb5kdc[3779](info): commencing operation
krb5kdc: starting...
W20260811 06:29:57.349411 29631 mini_kdc.cc:121] Time spent starting KDC: real 1.279s user 0.003s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/krb5kdc/test-user.keytab.
Aug 11 06:29:57 dist-test-slave-gkw7 krb5kdc[3779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429797, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:29:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:29:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.239.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.239.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.239.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab.
Entry for principal kudu/127.28.239.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab.
I20260811 06:29:57.593168 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:45063
--webserver_interface=127.28.239.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab
--principal=kudu/127.28.239.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:36419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.239.254:45063
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:29:57.954658 3803 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:29:57.955086 3803 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:29:57.955282 3803 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:29:57.962848 3803 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:29:57.963037 3803 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:29:57.963166 3803 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:29:57.963279 3803 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:29:57.963382 3803 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:29:57.973940 3803 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:36419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.239.254:45063
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab
--principal=kudu/127.28.239.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:45063
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.239.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:29:57.975484 3803 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:29:57.977054 3803 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:29:57.990430 3816 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:29:57.990438 3814 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:29:57.990695 3810 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:29:57.991357 3803 server_base.cc:1034] running on GCE node
I20260811 06:29:57.992261 3803 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:29:57.993841 3803 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:29:57.995060 3803 hybrid_clock.cc:648] HybridClock initialized: now 1786429797995006 us; error 44 us; skew 500 ppm
I20260811 06:29:57.995507 3803 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:29:57 dist-test-slave-gkw7 krb5kdc[3779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429797, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:29:57.998979 3803 init.cc:377] Logged in from keytab as kudu/127.28.239.254@KRBTEST.COM (short username kudu)
I20260811 06:29:58.000358 3803 fs_manager.cc:359] Metadata directory not provided
I20260811 06:29:58.000497 3803 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:29:58.000909 3803 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:29:58.004410 3803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/minicluster-data/master-0/data/instance:
uuid: "e95a4258059c432a9535729685e1f15e"
format_stamp: "Formatted at 2026-08-11 06:29:57 on dist-test-slave-gkw7"
server_key: "dbe93040ef9305f2785b8f5a8bfdba93"
server_key_iv: "231c815228662d904358e0b80ff7d7ee"
server_key_version: "encryptionkey@0"
I20260811 06:29:58.005342 3803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/minicluster-data/master-0/wal/instance:
uuid: "e95a4258059c432a9535729685e1f15e"
format_stamp: "Formatted at 2026-08-11 06:29:57 on dist-test-slave-gkw7"
server_key: "dbe93040ef9305f2785b8f5a8bfdba93"
server_key_iv: "231c815228662d904358e0b80ff7d7ee"
server_key_version: "encryptionkey@0"
I20260811 06:29:58.011154 3803 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 06:29:58.015167 3827 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:29:58.016800 3803 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260811 06:29:58.017045 3803 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/minicluster-data/master-0/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/minicluster-data/master-0/wal
uuid: "e95a4258059c432a9535729685e1f15e"
format_stamp: "Formatted at 2026-08-11 06:29:57 on dist-test-slave-gkw7"
server_key: "dbe93040ef9305f2785b8f5a8bfdba93"
server_key_iv: "231c815228662d904358e0b80ff7d7ee"
server_key_version: "encryptionkey@0"
I20260811 06:29:58.017338 3803 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:29:58.032012 3803 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:29:58.037145 3803 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:29:58.037474 3803 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:29:58.059167 3803 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.254:45063
I20260811 06:29:58.059211 3902 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.254:45063 every 8 connection(s)
I20260811 06:29:58.060264 3803 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
I20260811 06:29:58.064581 3904 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:29:58.068039 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 3803
I20260811 06:29:58.068307 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/minicluster-data/master-0/wal/instance
I20260811 06:29:58.069264 29631 external_mini_cluster.cc:1428] Setting key f1c31a6ac5b92fd85271a570a1d790b9
I20260811 06:29:58.079784 3904 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e95a4258059c432a9535729685e1f15e: Bootstrap starting.
I20260811 06:29:58.084481 3904 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e95a4258059c432a9535729685e1f15e: Neither blocks nor log segments found. Creating new log.
I20260811 06:29:58.086071 3904 log.cc:824] T 00000000000000000000000000000000 P e95a4258059c432a9535729685e1f15e: Log is configured to *not* fsync() on all Append() calls
Aug 11 06:29:58 dist-test-slave-gkw7 krb5kdc[3779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429797, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:29:58.090444 3904 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e95a4258059c432a9535729685e1f15e: No bootstrap required, opened a new log
I20260811 06:29:58.097819 3908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:58.071864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37046 (local address 127.28.239.254:45063)
0811 06:29:58.072827 (+ 963us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:58.072841 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:58.072890 (+ 49us) server_negotiation.cc:407] Connection header received
0811 06:29:58.074769 (+ 1879us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:58.074796 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:58.075251 (+ 455us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:58.075622 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:58.077847 (+ 2225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:58.079171 (+ 1324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:58.081205 (+ 2034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:58.082025 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:58.082175 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:58.087938 (+ 5763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:29:58.087951 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:29:58.087975 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:29:58.088024 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:29:58.092481 (+ 4457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:58.093265 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:58.093278 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:58.093294 (+ 16us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:58.093573 (+ 279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:58.094016 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:58.094022 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:58.094026 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:58.094908 (+ 882us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:29:58.095248 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:58.095645 (+ 397us) server_negotiation.cc:299] Negotiation successful
0811 06:29:58.096156 (+ 511us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":715,"thread_start_us":465,"threads_started":1}
I20260811 06:29:58.101341 3904 raft_consensus.cc:348] T 00000000000000000000000000000000 P e95a4258059c432a9535729685e1f15e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e95a4258059c432a9535729685e1f15e" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 45063 } }
I20260811 06:29:58.101883 3904 raft_consensus.cc:374] T 00000000000000000000000000000000 P e95a4258059c432a9535729685e1f15e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:29:58.101984 3904 raft_consensus.cc:729] T 00000000000000000000000000000000 P e95a4258059c432a9535729685e1f15e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e95a4258059c432a9535729685e1f15e, State: Initialized, Role: FOLLOWER
I20260811 06:29:58.102553 3904 consensus_queue.cc:260] T 00000000000000000000000000000000 P e95a4258059c432a9535729685e1f15e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e95a4258059c432a9535729685e1f15e" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 45063 } }
I20260811 06:29:58.102779 3904 raft_consensus.cc:388] T 00000000000000000000000000000000 P e95a4258059c432a9535729685e1f15e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:29:58.102893 3904 raft_consensus.cc:482] T 00000000000000000000000000000000 P e95a4258059c432a9535729685e1f15e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:29:58.103029 3904 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e95a4258059c432a9535729685e1f15e [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:29:58.105427 3904 raft_consensus.cc:504] T 00000000000000000000000000000000 P e95a4258059c432a9535729685e1f15e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e95a4258059c432a9535729685e1f15e" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 45063 } }
I20260811 06:29:58.106009 3904 leader_election.cc:302] T 00000000000000000000000000000000 P e95a4258059c432a9535729685e1f15e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e95a4258059c432a9535729685e1f15e; no voters:
I20260811 06:29:58.106705 3904 leader_election.cc:288] T 00000000000000000000000000000000 P e95a4258059c432a9535729685e1f15e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:29:58.106863 3910 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e95a4258059c432a9535729685e1f15e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:29:58.108091 3910 raft_consensus.cc:686] T 00000000000000000000000000000000 P e95a4258059c432a9535729685e1f15e [term 1 LEADER]: Becoming Leader. State: Replica: e95a4258059c432a9535729685e1f15e, State: Running, Role: LEADER
I20260811 06:29:58.108726 3910 consensus_queue.cc:237] T 00000000000000000000000000000000 P e95a4258059c432a9535729685e1f15e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e95a4258059c432a9535729685e1f15e" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 45063 } }
I20260811 06:29:58.109524 3904 sys_catalog.cc:564] T 00000000000000000000000000000000 P e95a4258059c432a9535729685e1f15e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:29:58.112126 3912 sys_catalog.cc:455] T 00000000000000000000000000000000 P e95a4258059c432a9535729685e1f15e [sys.catalog]: SysCatalogTable state changed. Reason: New leader e95a4258059c432a9535729685e1f15e. Latest consensus state: current_term: 1 leader_uuid: "e95a4258059c432a9535729685e1f15e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e95a4258059c432a9535729685e1f15e" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 45063 } } }
I20260811 06:29:58.112484 3912 sys_catalog.cc:458] T 00000000000000000000000000000000 P e95a4258059c432a9535729685e1f15e [sys.catalog]: This master's current role is: LEADER
I20260811 06:29:58.112980 3911 sys_catalog.cc:455] T 00000000000000000000000000000000 P e95a4258059c432a9535729685e1f15e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e95a4258059c432a9535729685e1f15e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e95a4258059c432a9535729685e1f15e" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 45063 } } }
I20260811 06:29:58.113335 3911 sys_catalog.cc:458] T 00000000000000000000000000000000 P e95a4258059c432a9535729685e1f15e [sys.catalog]: This master's current role is: LEADER
I20260811 06:29:58.116703 3918 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:29:58.122637 3918 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:29:58.131666 3918 catalog_manager.cc:1347] Generated new cluster ID: 9a9e7f3d0d5343e2a3e7362f8989c62f
I20260811 06:29:58.131816 3918 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:29:58.169025 3918 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:29:58.170945 3918 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:29:58.180744 3918 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e95a4258059c432a9535729685e1f15e: Generated new TSK 0
I20260811 06:29:58.181828 3918 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.239.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.239.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.239.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.193.keytab.
Entry for principal kudu/127.28.239.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.193.keytab.
I20260811 06:29:58.242100 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.193:0
--local_ip_for_outbound_sockets=127.28.239.193
--webserver_interface=127.28.239.193
--webserver_port=0
--tserver_master_addrs=127.28.239.254:45063
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.193.keytab
--principal=kudu/127.28.239.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:36419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:29:58.590193 3939 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:29:58.590471 3939 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:29:58.590570 3939 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:29:58.599936 3939 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:29:58.600131 3939 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:29:58.600332 3939 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.193
I20260811 06:29:58.611454 3939 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:36419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.193.keytab
--principal=kudu/127.28.239.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.239.193
--webserver_port=0
--tserver_master_addrs=127.28.239.254:45063
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.193
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:29:58.612743 3939 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:29:58.614307 3939 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:29:58.627712 3949 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:29:58.627797 3950 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:29:58.627766 3952 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:29:58.629061 3939 server_base.cc:1034] running on GCE node
I20260811 06:29:58.629719 3939 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:29:58.630609 3939 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:29:58.631871 3939 hybrid_clock.cc:648] HybridClock initialized: now 1786429798631819 us; error 60 us; skew 500 ppm
I20260811 06:29:58.632221 3939 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:29:58 dist-test-slave-gkw7 krb5kdc[3779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429798, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:29:58.635651 3939 init.cc:377] Logged in from keytab as kudu/127.28.239.193@KRBTEST.COM (short username kudu)
I20260811 06:29:58.636947 3939 fs_manager.cc:359] Metadata directory not provided
I20260811 06:29:58.637066 3939 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:29:58.637406 3939 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:29:58.640718 3939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/minicluster-data/ts-0/data/instance:
uuid: "27607b5859494d26915b1ce2e2c22774"
format_stamp: "Formatted at 2026-08-11 06:29:58 on dist-test-slave-gkw7"
server_key: "ba8f0f7127d00be8cb1c329cbbc0185b"
server_key_iv: "1599539f1ecd9616b07d15203c40882a"
server_key_version: "encryptionkey@0"
I20260811 06:29:58.641737 3939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/minicluster-data/ts-0/wal/instance:
uuid: "27607b5859494d26915b1ce2e2c22774"
format_stamp: "Formatted at 2026-08-11 06:29:58 on dist-test-slave-gkw7"
server_key: "ba8f0f7127d00be8cb1c329cbbc0185b"
server_key_iv: "1599539f1ecd9616b07d15203c40882a"
server_key_version: "encryptionkey@0"
I20260811 06:29:58.647217 3939 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 06:29:58.650867 3963 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:29:58.652192 3939 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 06:29:58.652380 3939 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/minicluster-data/ts-0/wal
uuid: "27607b5859494d26915b1ce2e2c22774"
format_stamp: "Formatted at 2026-08-11 06:29:58 on dist-test-slave-gkw7"
server_key: "ba8f0f7127d00be8cb1c329cbbc0185b"
server_key_iv: "1599539f1ecd9616b07d15203c40882a"
server_key_version: "encryptionkey@0"
I20260811 06:29:58.652611 3939 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:29:58.675444 3939 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:29:58.679863 3939 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:29:58.680158 3939 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:29:58.681320 3939 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:29:58.682740 3939 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:29:58.682843 3939 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:29:58.682924 3939 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:29:58.682987 3939 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:29:58.717948 3939 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.193:33079
I20260811 06:29:58.718009 4133 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.193:33079 every 8 connection(s)
I20260811 06:29:58.718880 3939 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
I20260811 06:29:58.727035 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 3939
I20260811 06:29:58.727335 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/minicluster-data/ts-0/wal/instance
I20260811 06:29:58.727993 29631 external_mini_cluster.cc:1428] Setting key 90a5255b0dfa21c2e13618b691ea3271
Aug 11 06:29:58 dist-test-slave-gkw7 krb5kdc[3779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429798, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.193@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:29:58.741040 4140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:58.721824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.193:35233 (local address 127.28.239.254:45063)
0811 06:29:58.722379 (+ 555us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:58.722384 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:58.723497 (+ 1113us) server_negotiation.cc:407] Connection header received
0811 06:29:58.725429 (+ 1932us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:58.725435 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:58.725585 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:58.725718 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:58.727401 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:58.728231 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:58.730240 (+ 2009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:58.730672 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:58.730815 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:58.734842 (+ 4027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:29:58.734847 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:29:58.734851 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:29:58.734884 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:29:58.737476 (+ 2592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:58.738330 (+ 854us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:58.738338 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:58.738342 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:58.738507 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:58.738970 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:58.738977 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:58.738982 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:58.739408 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:29:58.739567 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:58.740286 (+ 719us) server_negotiation.cc:299] Negotiation successful
0811 06:29:58.740552 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":220,"threads_started":1}
I20260811 06:29:58.741745 4142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:58.722562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:45063 (local address 127.28.239.193:35233)
0811 06:29:58.723327 (+ 765us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:58.723368 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 06:29:58.725138 (+ 1770us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:29:58.725984 (+ 846us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:29:58.725993 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:29:58.726549 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:29:58.727213 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:58.727246 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:58.728423 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:58.728427 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:58.730080 (+ 1653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:58.730096 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:58.730931 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:58.730939 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:58.731198 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:58.731934 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:29:58.731958 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:29:58.734588 (+ 2630us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:29:58.737691 (+ 3103us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:29:58.737702 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:29:58.737724 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:29:58.738181 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:29:58.738622 (+ 441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:29:58.738625 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:29:58.738627 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:29:58.738842 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:29:58.739532 (+ 690us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:29:58.739539 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:29:58.739880 (+ 341us) client_negotiation.cc:769] Sending connection context
0811 06:29:58.740107 (+ 227us) client_negotiation.cc:240] Negotiation successful
0811 06:29:58.740508 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":286,"threads_started":1}
I20260811 06:29:58.743427 4134 heartbeater.cc:347] Connected to a master server at 127.28.239.254:45063
I20260811 06:29:58.743835 4134 heartbeater.cc:464] Registering TS with master...
I20260811 06:29:58.744664 4134 heartbeater.cc:511] Master 127.28.239.254:45063 requested a full tablet report, sending...
I20260811 06:29:58.747079 3851 ts_manager.cc:194] Registered new tserver with Master: 27607b5859494d26915b1ce2e2c22774 (127.28.239.193:33079)
I20260811 06:29:58.748546 3851 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.239.193@KRBTEST.COM'} at 127.28.239.193:35233
WARNING: no policy specified for kudu/127.28.239.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.239.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.239.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.194.keytab.
Entry for principal kudu/127.28.239.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.194.keytab.
I20260811 06:29:58.784906 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.194:0
--local_ip_for_outbound_sockets=127.28.239.194
--webserver_interface=127.28.239.194
--webserver_port=0
--tserver_master_addrs=127.28.239.254:45063
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.194.keytab
--principal=kudu/127.28.239.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:36419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:29:59.139664 4148 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:29:59.139974 4148 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:29:59.140096 4148 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:29:59.147789 4148 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:29:59.147927 4148 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:29:59.148074 4148 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.194
I20260811 06:29:59.158833 4148 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:36419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.194.keytab
--principal=kudu/127.28.239.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.239.194
--webserver_port=0
--tserver_master_addrs=127.28.239.254:45063
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.194
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:29:59.160247 4148 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:29:59.161733 4148 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:29:59.175526 4157 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:29:59.175630 4156 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:29:59.176236 4159 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:29:59.177018 4148 server_base.cc:1034] running on GCE node
I20260811 06:29:59.177711 4148 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:29:59.178699 4148 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:29:59.179978 4148 hybrid_clock.cc:648] HybridClock initialized: now 1786429799179920 us; error 49 us; skew 500 ppm
I20260811 06:29:59.180318 4148 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:29:59 dist-test-slave-gkw7 krb5kdc[3779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429799, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:29:59.187403 4148 init.cc:377] Logged in from keytab as kudu/127.28.239.194@KRBTEST.COM (short username kudu)
I20260811 06:29:59.188695 4148 fs_manager.cc:359] Metadata directory not provided
I20260811 06:29:59.188848 4148 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:29:59.189198 4148 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:29:59.192381 4148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/minicluster-data/ts-1/data/instance:
uuid: "c3f60d945e1a4ea78490908fd6a4f8a0"
format_stamp: "Formatted at 2026-08-11 06:29:59 on dist-test-slave-gkw7"
server_key: "62f96edaa54bd2bcbd8517e993051533"
server_key_iv: "e8bde261930bc0728d879cb6f4a82aa1"
server_key_version: "encryptionkey@0"
I20260811 06:29:59.193231 4148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/minicluster-data/ts-1/wal/instance:
uuid: "c3f60d945e1a4ea78490908fd6a4f8a0"
format_stamp: "Formatted at 2026-08-11 06:29:59 on dist-test-slave-gkw7"
server_key: "62f96edaa54bd2bcbd8517e993051533"
server_key_iv: "e8bde261930bc0728d879cb6f4a82aa1"
server_key_version: "encryptionkey@0"
I20260811 06:29:59.198612 4148 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260811 06:29:59.202175 4169 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:29:59.203514 4148 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 06:29:59.203706 4148 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/minicluster-data/ts-1/wal
uuid: "c3f60d945e1a4ea78490908fd6a4f8a0"
format_stamp: "Formatted at 2026-08-11 06:29:59 on dist-test-slave-gkw7"
server_key: "62f96edaa54bd2bcbd8517e993051533"
server_key_iv: "e8bde261930bc0728d879cb6f4a82aa1"
server_key_version: "encryptionkey@0"
I20260811 06:29:59.203965 4148 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:29:59.226584 4148 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:29:59.232264 4148 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:29:59.232582 4148 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:29:59.233953 4148 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:29:59.235916 4148 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:29:59.236083 4148 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:29:59.236197 4148 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:29:59.236280 4148 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:29:59.286300 4148 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.194:35731
I20260811 06:29:59.286372 4349 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.194:35731 every 8 connection(s)
I20260811 06:29:59.287552 4148 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
I20260811 06:29:59.290262 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 4148
I20260811 06:29:59.290472 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/minicluster-data/ts-1/wal/instance
I20260811 06:29:59.291042 29631 external_mini_cluster.cc:1428] Setting key 48d344f08f61f89697af3dc3b92f3f19
Aug 11 06:29:59 dist-test-slave-gkw7 krb5kdc[3779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429799, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.194@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:29:59.314313 4356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:59.291871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.194:49341 (local address 127.28.239.254:45063)
0811 06:29:59.292348 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:59.292357 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:59.295894 (+ 3537us) server_negotiation.cc:407] Connection header received
0811 06:29:59.295984 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:59.295992 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:59.296112 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:59.296250 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:59.298985 (+ 2735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:59.300225 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:59.302215 (+ 1990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:59.303003 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:59.303122 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:59.307825 (+ 4703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:29:59.307831 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:29:59.307835 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:29:59.307869 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:29:59.310786 (+ 2917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:59.311564 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:59.311569 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:59.311578 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:59.311750 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:59.312359 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:59.312363 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:59.312365 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:59.312638 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:29:59.312754 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:59.313844 (+ 1090us) server_negotiation.cc:299] Negotiation successful
0811 06:29:59.314025 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":221,"threads_started":1}
I20260811 06:29:59.316102 4353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:59.291240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:45063 (local address 127.28.239.194:49341)
0811 06:29:59.292256 (+ 1016us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:59.292310 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 06:29:59.295447 (+ 3137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:29:59.296766 (+ 1319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:29:59.296781 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:29:59.297744 (+ 963us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:29:59.298713 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:59.298748 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:59.300483 (+ 1735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:59.300488 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:59.302002 (+ 1514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:59.302016 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:59.303276 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:59.303286 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:59.303640 (+ 354us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:59.304468 (+ 828us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:29:59.304507 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:29:59.307497 (+ 2990us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:29:59.310926 (+ 3429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:29:59.310936 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:29:59.310954 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:29:59.311425 (+ 471us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:29:59.311921 (+ 496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:29:59.311930 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:29:59.311936 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:29:59.312222 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:29:59.312793 (+ 571us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:29:59.312806 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:29:59.313349 (+ 543us) client_negotiation.cc:769] Sending connection context
0811 06:29:59.313764 (+ 415us) client_negotiation.cc:240] Negotiation successful
0811 06:29:59.314416 (+ 652us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":651,"thread_start_us":410,"threads_started":1}
WARNING: no policy specified for kudu/127.28.239.195@KRBTEST.COM; defaulting to no policy
I20260811 06:29:59.317824 4350 heartbeater.cc:347] Connected to a master server at 127.28.239.254:45063
I20260811 06:29:59.318226 4350 heartbeater.cc:464] Registering TS with master...
I20260811 06:29:59.318965 4350 heartbeater.cc:511] Master 127.28.239.254:45063 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.239.195@KRBTEST.COM" created.
I20260811 06:29:59.320683 3851 ts_manager.cc:194] Registered new tserver with Master: c3f60d945e1a4ea78490908fd6a4f8a0 (127.28.239.194:35731)
I20260811 06:29:59.322011 3851 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.239.194@KRBTEST.COM'} at 127.28.239.194:49341
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.239.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.195.keytab.
Entry for principal kudu/127.28.239.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.195.keytab.
I20260811 06:29:59.348899 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.195:0
--local_ip_for_outbound_sockets=127.28.239.195
--webserver_interface=127.28.239.195
--webserver_port=0
--tserver_master_addrs=127.28.239.254:45063
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.195.keytab
--principal=kudu/127.28.239.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:36419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:29:59.696379 4359 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:29:59.696728 4359 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:29:59.696913 4359 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:29:59.704336 4359 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:29:59.704499 4359 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:29:59.704679 4359 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.195
I20260811 06:29:59.715029 4359 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:36419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.195.keytab
--principal=kudu/127.28.239.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.239.195
--webserver_port=0
--tserver_master_addrs=127.28.239.254:45063
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.195
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:29:59.716430 4359 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:29:59.717955 4359 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:29:59.731731 4367 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:29:59.731724 4366 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:29:59.731938 4370 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:29:59.732678 4359 server_base.cc:1034] running on GCE node
I20260811 06:29:59.733346 4359 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:29:59.734443 4359 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:29:59.735657 4359 hybrid_clock.cc:648] HybridClock initialized: now 1786429799735580 us; error 66 us; skew 500 ppm
I20260811 06:29:59.736016 4359 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:29:59 dist-test-slave-gkw7 krb5kdc[3779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429799, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:29:59.739539 4359 init.cc:377] Logged in from keytab as kudu/127.28.239.195@KRBTEST.COM (short username kudu)
I20260811 06:29:59.740792 4359 fs_manager.cc:359] Metadata directory not provided
I20260811 06:29:59.740936 4359 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:29:59.741293 4359 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:29:59.744539 4359 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/minicluster-data/ts-2/data/instance:
uuid: "5ab6a4d25dbb46378021701be3bb0fd6"
format_stamp: "Formatted at 2026-08-11 06:29:59 on dist-test-slave-gkw7"
server_key: "6505df7f9dc1d21fe057df0be5104697"
server_key_iv: "41f0cc604e0dd5a471697eb237929d2f"
server_key_version: "encryptionkey@0"
I20260811 06:29:59.745401 4359 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/minicluster-data/ts-2/wal/instance:
uuid: "5ab6a4d25dbb46378021701be3bb0fd6"
format_stamp: "Formatted at 2026-08-11 06:29:59 on dist-test-slave-gkw7"
server_key: "6505df7f9dc1d21fe057df0be5104697"
server_key_iv: "41f0cc604e0dd5a471697eb237929d2f"
server_key_version: "encryptionkey@0"
I20260811 06:29:59.751034 4359 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260811 06:29:59.752399 4134 heartbeater.cc:503] Master 127.28.239.254:45063 was elected leader, sending a full tablet report...
I20260811 06:29:59.755131 4378 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:29:59.756660 4359 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 06:29:59.756850 4359 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/minicluster-data/ts-2/wal
uuid: "5ab6a4d25dbb46378021701be3bb0fd6"
format_stamp: "Formatted at 2026-08-11 06:29:59 on dist-test-slave-gkw7"
server_key: "6505df7f9dc1d21fe057df0be5104697"
server_key_iv: "41f0cc604e0dd5a471697eb237929d2f"
server_key_version: "encryptionkey@0"
I20260811 06:29:59.757097 4359 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:29:59.786803 4359 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:29:59.791687 4359 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:29:59.792004 4359 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:29:59.793098 4359 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:29:59.794703 4359 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:29:59.794829 4359 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:29:59.794960 4359 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:29:59.795054 4359 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:29:59.830364 4359 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.195:46713
I20260811 06:29:59.830442 4572 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.195:46713 every 8 connection(s)
I20260811 06:29:59.831298 4359 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
I20260811 06:29:59.833937 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 4359
I20260811 06:29:59.834149 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/minicluster-data/ts-2/wal/instance
I20260811 06:29:59.834735 29631 external_mini_cluster.cc:1428] Setting key 4f2ff555b7ebf835ca7df521cf3a6cbd
Aug 11 06:29:59 dist-test-slave-gkw7 krb5kdc[3779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429799, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.195@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:29:59.852171 4576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:59.833993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.195:46711 (local address 127.28.239.254:45063)
0811 06:29:59.834608 (+ 615us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:59.834615 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:59.835699 (+ 1084us) server_negotiation.cc:407] Connection header received
0811 06:29:59.838135 (+ 2436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:59.838139 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:59.838218 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:59.838314 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:59.839792 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:59.840570 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:59.842177 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:59.842604 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:59.842702 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:59.846080 (+ 3378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:29:59.846085 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:29:59.846088 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:29:59.846122 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:29:59.848907 (+ 2785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:59.849905 (+ 998us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:59.849910 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:59.849912 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:59.850086 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:59.850638 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:59.850641 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:59.850644 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:59.850922 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:29:59.851044 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:59.851697 (+ 653us) server_negotiation.cc:299] Negotiation successful
0811 06:29:59.851889 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":476,"thread_start_us":325,"threads_started":1}
I20260811 06:29:59.853127 4577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:59.834620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:45063 (local address 127.28.239.195:46711)
0811 06:29:59.835476 (+ 856us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:59.835544 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0811 06:29:59.837813 (+ 2269us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:29:59.838504 (+ 691us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:29:59.838512 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:29:59.839033 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:29:59.839627 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:59.839644 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:59.840694 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:59.840697 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:59.842048 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:59.842060 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:59.842794 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:59.842798 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:59.842985 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:59.843625 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:29:59.843647 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:29:59.845882 (+ 2235us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:29:59.849065 (+ 3183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:29:59.849072 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:29:59.849083 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:29:59.849746 (+ 663us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:29:59.850208 (+ 462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:29:59.850211 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:29:59.850214 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:29:59.850485 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:29:59.851042 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:29:59.851048 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:29:59.851367 (+ 319us) client_negotiation.cc:769] Sending connection context
0811 06:29:59.851590 (+ 223us) client_negotiation.cc:240] Negotiation successful
0811 06:29:59.851987 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":566,"thread_start_us":348,"threads_started":1}
I20260811 06:29:59.854665 4573 heartbeater.cc:347] Connected to a master server at 127.28.239.254:45063
I20260811 06:29:59.854986 4573 heartbeater.cc:464] Registering TS with master...
I20260811 06:29:59.855597 4573 heartbeater.cc:511] Master 127.28.239.254:45063 requested a full tablet report, sending...
I20260811 06:29:59.857154 3851 ts_manager.cc:194] Registered new tserver with Master: 5ab6a4d25dbb46378021701be3bb0fd6 (127.28.239.195:46713)
I20260811 06:29:59.858045 3851 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.239.195@KRBTEST.COM'} at 127.28.239.195:46711
I20260811 06:29:59.862990 29631 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 06:29:59.880105 4576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:59.867687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37062 (local address 127.28.239.254:45063)
0811 06:29:59.867866 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:59.867871 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:59.868426 (+ 555us) server_negotiation.cc:407] Connection header received
0811 06:29:59.868573 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:59.868577 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:59.868642 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:59.868727 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:29:59.870257 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:59.870992 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:59.872274 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:59.872689 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:59.872793 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:59.875207 (+ 2414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:29:59.875212 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:29:59.875215 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:29:59.875249 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:29:59.877700 (+ 2451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:59.878325 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:59.878329 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:59.878331 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:59.878452 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:29:59.878783 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:29:59.878786 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:29:59.878788 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:29:59.879109 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:29:59.879297 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:59.879655 (+ 358us) server_negotiation.cc:299] Negotiation successful
0811 06:29:59.879853 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 06:29:59.884649 3851 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37062:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:29:59.891650 3851 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:29:59.917402 4604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:59.908477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36264 (local address 127.28.239.193:33079)
0811 06:29:59.909189 (+ 712us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:59.909197 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:59.909221 (+ 24us) server_negotiation.cc:407] Connection header received
0811 06:29:59.909372 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:59.909381 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:59.909633 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:59.909848 (+ 215us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:29:59.910576 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:59.911866 (+ 1290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:59.914175 (+ 2309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:59.915710 (+ 1535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:59.915938 (+ 228us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:59.916141 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:59.916838 (+ 697us) server_negotiation.cc:299] Negotiation successful
0811 06:29:59.917019 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":548,"thread_start_us":386,"threads_started":1}
I20260811 06:29:59.917465 4597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:59.905681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60908 (local address 127.28.239.194:35731)
0811 06:29:59.907715 (+ 2034us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:59.907725 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:59.907788 (+ 63us) server_negotiation.cc:407] Connection header received
0811 06:29:59.907944 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:59.907954 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:59.908962 (+ 1008us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:59.909198 (+ 236us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:29:59.909939 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:59.911569 (+ 1630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:59.914505 (+ 2936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:59.915967 (+ 1462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:59.916133 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:59.916321 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:59.916944 (+ 623us) server_negotiation.cc:299] Negotiation successful
0811 06:29:59.917124 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1716,"thread_start_us":1507,"threads_started":1}
I20260811 06:29:59.917496 4600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:59.907815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.193:33079 (local address 127.0.0.1:36264)
0811 06:29:59.908447 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:59.908467 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 06:29:59.908593 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:29:59.909846 (+ 1253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:29:59.909850 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:29:59.909871 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:29:59.910387 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:59.910401 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:59.912037 (+ 1636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:59.912042 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:59.914008 (+ 1966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:59.914022 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:59.916301 (+ 2279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:59.916308 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:59.916541 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:59.916610 (+ 69us) client_negotiation.cc:769] Sending connection context
0811 06:29:59.916766 (+ 156us) client_negotiation.cc:240] Negotiation successful
0811 06:29:59.917001 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"mutex_wait_us":124,"thread_start_us":334,"threads_started":1}
I20260811 06:29:59.918406 4596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:59.905066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54712 (local address 127.28.239.195:46713)
0811 06:29:59.905974 (+ 908us) server_negotiation.cc:206] Beginning negotiation
0811 06:29:59.905982 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:29:59.908699 (+ 2717us) server_negotiation.cc:407] Connection header received
0811 06:29:59.908811 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:29:59.908819 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:29:59.909046 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:29:59.909210 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:29:59.910505 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:59.912051 (+ 1546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:59.914691 (+ 2640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:29:59.916089 (+ 1398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:29:59.916306 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:59.916472 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0811 06:29:59.918036 (+ 1564us) server_negotiation.cc:299] Negotiation successful
0811 06:29:59.918166 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":718,"thread_start_us":771,"threads_started":1}
I20260811 06:29:59.918716 4594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:59.904937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.195:46713 (local address 127.0.0.1:54712)
0811 06:29:59.908436 (+ 3499us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:59.908555 (+ 119us) client_negotiation.cc:174] Beginning negotiation
0811 06:29:59.908681 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:29:59.909521 (+ 840us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:29:59.909524 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:29:59.909654 (+ 130us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:29:59.910221 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:59.910239 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:59.912234 (+ 1995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:59.912239 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:59.914533 (+ 2294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:59.914543 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:59.916643 (+ 2100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:59.916648 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:59.916878 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:59.916912 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 06:29:59.916988 (+ 76us) client_negotiation.cc:240] Negotiation successful
0811 06:29:59.917111 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2471,"mutex_wait_us":26,"thread_start_us":2495,"threads_started":1}
I20260811 06:29:59.919203 4593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:29:59.904542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.194:35731 (local address 127.0.0.1:60908)
0811 06:29:59.906696 (+ 2154us) negotiation.cc:107] Waiting for socket to connect
0811 06:29:59.906758 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 06:29:59.906972 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:29:59.909199 (+ 2227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:29:59.909206 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:29:59.909242 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:29:59.909705 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:59.909722 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:59.911746 (+ 2024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:59.911752 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:59.914309 (+ 2557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:29:59.914324 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:29:59.916470 (+ 2146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:29:59.916476 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:29:59.916711 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:29:59.916761 (+ 50us) client_negotiation.cc:769] Sending connection context
0811 06:29:59.916826 (+ 65us) client_negotiation.cc:240] Negotiation successful
0811 06:29:59.918246 (+ 1420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1804,"mutex_wait_us":1308,"thread_start_us":1598,"threads_started":1}
I20260811 06:29:59.921995 4035 tablet_service.cc:1467] Processing CreateTablet for tablet 0c3fc01340094a50b4178a9954662d8c (DEFAULT_TABLE table=test-table [id=e29671a0ab304d41a78539596ecabeda]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:29:59.922360 4452 tablet_service.cc:1467] Processing CreateTablet for tablet 0c3fc01340094a50b4178a9954662d8c (DEFAULT_TABLE table=test-table [id=e29671a0ab304d41a78539596ecabeda]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:29:59.922705 4246 tablet_service.cc:1467] Processing CreateTablet for tablet 0c3fc01340094a50b4178a9954662d8c (DEFAULT_TABLE table=test-table [id=e29671a0ab304d41a78539596ecabeda]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:29:59.923476 4035 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c3fc01340094a50b4178a9954662d8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:29:59.923732 4452 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c3fc01340094a50b4178a9954662d8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:29:59.923931 4246 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c3fc01340094a50b4178a9954662d8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:29:59.936964 4605 tablet_bootstrap.cc:492] T 0c3fc01340094a50b4178a9954662d8c P 27607b5859494d26915b1ce2e2c22774: Bootstrap starting.
I20260811 06:29:59.940407 4605 tablet_bootstrap.cc:654] T 0c3fc01340094a50b4178a9954662d8c P 27607b5859494d26915b1ce2e2c22774: Neither blocks nor log segments found. Creating new log.
I20260811 06:29:59.941402 4607 tablet_bootstrap.cc:492] T 0c3fc01340094a50b4178a9954662d8c P c3f60d945e1a4ea78490908fd6a4f8a0: Bootstrap starting.
I20260811 06:29:59.942003 4605 log.cc:824] T 0c3fc01340094a50b4178a9954662d8c P 27607b5859494d26915b1ce2e2c22774: Log is configured to *not* fsync() on all Append() calls
I20260811 06:29:59.942441 4608 tablet_bootstrap.cc:492] T 0c3fc01340094a50b4178a9954662d8c P 5ab6a4d25dbb46378021701be3bb0fd6: Bootstrap starting.
I20260811 06:29:59.946182 4607 tablet_bootstrap.cc:654] T 0c3fc01340094a50b4178a9954662d8c P c3f60d945e1a4ea78490908fd6a4f8a0: Neither blocks nor log segments found. Creating new log.
I20260811 06:29:59.946728 4608 tablet_bootstrap.cc:654] T 0c3fc01340094a50b4178a9954662d8c P 5ab6a4d25dbb46378021701be3bb0fd6: Neither blocks nor log segments found. Creating new log.
I20260811 06:29:59.947520 4607 log.cc:824] T 0c3fc01340094a50b4178a9954662d8c P c3f60d945e1a4ea78490908fd6a4f8a0: Log is configured to *not* fsync() on all Append() calls
I20260811 06:29:59.947957 4608 log.cc:824] T 0c3fc01340094a50b4178a9954662d8c P 5ab6a4d25dbb46378021701be3bb0fd6: Log is configured to *not* fsync() on all Append() calls
I20260811 06:29:59.948695 4605 tablet_bootstrap.cc:492] T 0c3fc01340094a50b4178a9954662d8c P 27607b5859494d26915b1ce2e2c22774: No bootstrap required, opened a new log
I20260811 06:29:59.949031 4605 ts_tablet_manager.cc:1397] T 0c3fc01340094a50b4178a9954662d8c P 27607b5859494d26915b1ce2e2c22774: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.003s
I20260811 06:29:59.950743 4608 tablet_bootstrap.cc:492] T 0c3fc01340094a50b4178a9954662d8c P 5ab6a4d25dbb46378021701be3bb0fd6: No bootstrap required, opened a new log
I20260811 06:29:59.950925 4607 tablet_bootstrap.cc:492] T 0c3fc01340094a50b4178a9954662d8c P c3f60d945e1a4ea78490908fd6a4f8a0: No bootstrap required, opened a new log
I20260811 06:29:59.951051 4608 ts_tablet_manager.cc:1397] T 0c3fc01340094a50b4178a9954662d8c P 5ab6a4d25dbb46378021701be3bb0fd6: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260811 06:29:59.951198 4607 ts_tablet_manager.cc:1397] T 0c3fc01340094a50b4178a9954662d8c P c3f60d945e1a4ea78490908fd6a4f8a0: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260811 06:29:59.959039 4608 raft_consensus.cc:348] T 0c3fc01340094a50b4178a9954662d8c P 5ab6a4d25dbb46378021701be3bb0fd6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3f60d945e1a4ea78490908fd6a4f8a0" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 35731 } } peers { permanent_uuid: "5ab6a4d25dbb46378021701be3bb0fd6" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 46713 } } peers { permanent_uuid: "27607b5859494d26915b1ce2e2c22774" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 33079 } }
I20260811 06:29:59.959465 4608 raft_consensus.cc:374] T 0c3fc01340094a50b4178a9954662d8c P 5ab6a4d25dbb46378021701be3bb0fd6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:29:59.959568 4608 raft_consensus.cc:729] T 0c3fc01340094a50b4178a9954662d8c P 5ab6a4d25dbb46378021701be3bb0fd6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ab6a4d25dbb46378021701be3bb0fd6, State: Initialized, Role: FOLLOWER
I20260811 06:29:59.960249 4608 consensus_queue.cc:260] T 0c3fc01340094a50b4178a9954662d8c P 5ab6a4d25dbb46378021701be3bb0fd6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3f60d945e1a4ea78490908fd6a4f8a0" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 35731 } } peers { permanent_uuid: "5ab6a4d25dbb46378021701be3bb0fd6" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 46713 } } peers { permanent_uuid: "27607b5859494d26915b1ce2e2c22774" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 33079 } }
I20260811 06:29:59.960255 4605 raft_consensus.cc:348] T 0c3fc01340094a50b4178a9954662d8c P 27607b5859494d26915b1ce2e2c22774 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3f60d945e1a4ea78490908fd6a4f8a0" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 35731 } } peers { permanent_uuid: "5ab6a4d25dbb46378021701be3bb0fd6" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 46713 } } peers { permanent_uuid: "27607b5859494d26915b1ce2e2c22774" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 33079 } }
I20260811 06:29:59.960731 4605 raft_consensus.cc:374] T 0c3fc01340094a50b4178a9954662d8c P 27607b5859494d26915b1ce2e2c22774 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:29:59.960850 4605 raft_consensus.cc:729] T 0c3fc01340094a50b4178a9954662d8c P 27607b5859494d26915b1ce2e2c22774 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27607b5859494d26915b1ce2e2c22774, State: Initialized, Role: FOLLOWER
I20260811 06:29:59.961599 4605 consensus_queue.cc:260] T 0c3fc01340094a50b4178a9954662d8c P 27607b5859494d26915b1ce2e2c22774 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3f60d945e1a4ea78490908fd6a4f8a0" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 35731 } } peers { permanent_uuid: "5ab6a4d25dbb46378021701be3bb0fd6" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 46713 } } peers { permanent_uuid: "27607b5859494d26915b1ce2e2c22774" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 33079 } }
I20260811 06:29:59.962023 4607 raft_consensus.cc:348] T 0c3fc01340094a50b4178a9954662d8c P c3f60d945e1a4ea78490908fd6a4f8a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3f60d945e1a4ea78490908fd6a4f8a0" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 35731 } } peers { permanent_uuid: "5ab6a4d25dbb46378021701be3bb0fd6" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 46713 } } peers { permanent_uuid: "27607b5859494d26915b1ce2e2c22774" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 33079 } }
I20260811 06:29:59.962519 4573 heartbeater.cc:503] Master 127.28.239.254:45063 was elected leader, sending a full tablet report...
I20260811 06:29:59.962558 4607 raft_consensus.cc:374] T 0c3fc01340094a50b4178a9954662d8c P c3f60d945e1a4ea78490908fd6a4f8a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:29:59.962679 4607 raft_consensus.cc:729] T 0c3fc01340094a50b4178a9954662d8c P c3f60d945e1a4ea78490908fd6a4f8a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3f60d945e1a4ea78490908fd6a4f8a0, State: Initialized, Role: FOLLOWER
I20260811 06:29:59.963086 4608 ts_tablet_manager.cc:1428] T 0c3fc01340094a50b4178a9954662d8c P 5ab6a4d25dbb46378021701be3bb0fd6: Time spent starting tablet: real 0.012s user 0.008s sys 0.004s
I20260811 06:29:59.963361 4607 consensus_queue.cc:260] T 0c3fc01340094a50b4178a9954662d8c P c3f60d945e1a4ea78490908fd6a4f8a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3f60d945e1a4ea78490908fd6a4f8a0" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 35731 } } peers { permanent_uuid: "5ab6a4d25dbb46378021701be3bb0fd6" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 46713 } } peers { permanent_uuid: "27607b5859494d26915b1ce2e2c22774" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 33079 } }
I20260811 06:29:59.970350 4605 ts_tablet_manager.cc:1428] T 0c3fc01340094a50b4178a9954662d8c P 27607b5859494d26915b1ce2e2c22774: Time spent starting tablet: real 0.021s user 0.018s sys 0.005s
I20260811 06:29:59.970412 4350 heartbeater.cc:503] Master 127.28.239.254:45063 was elected leader, sending a full tablet report...
I20260811 06:29:59.971136 4607 ts_tablet_manager.cc:1428] T 0c3fc01340094a50b4178a9954662d8c P c3f60d945e1a4ea78490908fd6a4f8a0: Time spent starting tablet: real 0.020s user 0.012s sys 0.008s
W20260811 06:29:59.971510 4136 tablet.cc:2369] T 0c3fc01340094a50b4178a9954662d8c P 27607b5859494d26915b1ce2e2c22774: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:30:00.035946 4615 raft_consensus.cc:482] T 0c3fc01340094a50b4178a9954662d8c P 27607b5859494d26915b1ce2e2c22774 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:30:00.036286 4615 raft_consensus.cc:504] T 0c3fc01340094a50b4178a9954662d8c P 27607b5859494d26915b1ce2e2c22774 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3f60d945e1a4ea78490908fd6a4f8a0" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 35731 } } peers { permanent_uuid: "5ab6a4d25dbb46378021701be3bb0fd6" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 46713 } } peers { permanent_uuid: "27607b5859494d26915b1ce2e2c22774" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 33079 } }
I20260811 06:30:00.038323 4615 leader_election.cc:288] T 0c3fc01340094a50b4178a9954662d8c P 27607b5859494d26915b1ce2e2c22774 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c3f60d945e1a4ea78490908fd6a4f8a0 (127.28.239.194:35731), 5ab6a4d25dbb46378021701be3bb0fd6 (127.28.239.195:46713)
W20260811 06:30:00.039691 4352 tablet.cc:2369] T 0c3fc01340094a50b4178a9954662d8c P c3f60d945e1a4ea78490908fd6a4f8a0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:30:00.046489 4597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:00.038726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.193:55055 (local address 127.28.239.194:35731)
0811 06:30:00.038970 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:00.038977 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:00.040014 (+ 1037us) server_negotiation.cc:407] Connection header received
0811 06:30:00.040235 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:00.040239 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:00.040300 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:00.040403 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:30:00.041026 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:00.041852 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:00.043865 (+ 2013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:00.045095 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:00.045288 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:00.045364 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:00.046004 (+ 640us) server_negotiation.cc:299] Negotiation successful
0811 06:30:00.046185 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 06:30:00.046483 4596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:00.038954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.193:55453 (local address 127.28.239.195:46713)
0811 06:30:00.039156 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:00.039161 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:00.039716 (+ 555us) server_negotiation.cc:407] Connection header received
0811 06:30:00.040075 (+ 359us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:00.040084 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:00.040170 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:00.040240 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:30:00.041155 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:00.041964 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:00.043932 (+ 1968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:00.045173 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:00.045321 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:00.045399 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:00.046004 (+ 605us) server_negotiation.cc:299] Negotiation successful
0811 06:30:00.046155 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 06:30:00.046617 4619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:00.038716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.194:35731 (local address 127.28.239.193:55055)
0811 06:30:00.039909 (+ 1193us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:00.039927 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:00.040067 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:00.040422 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:00.040427 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:00.040452 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:30:00.040846 (+ 394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:00.040860 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:00.041981 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:00.041989 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:00.043691 (+ 1702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:00.043705 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:00.045394 (+ 1689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:00.045406 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:00.045710 (+ 304us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:00.045810 (+ 100us) client_negotiation.cc:769] Sending connection context
0811 06:30:00.045919 (+ 109us) client_negotiation.cc:240] Negotiation successful
0811 06:30:00.046111 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1038,"mutex_wait_us":505,"thread_start_us":485,"threads_started":1}
I20260811 06:30:00.046864 4617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:00.038710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.195:46713 (local address 127.28.239.193:55453)
0811 06:30:00.039573 (+ 863us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:00.039598 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:00.039867 (+ 269us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:00.040479 (+ 612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:00.040484 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:00.040519 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:30:00.040994 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:00.041005 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:00.042121 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:00.042128 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:00.043720 (+ 1592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:00.043782 (+ 62us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:00.045421 (+ 1639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:00.045428 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:00.045662 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:00.045821 (+ 159us) client_negotiation.cc:769] Sending connection context
0811 06:30:00.045919 (+ 98us) client_negotiation.cc:240] Negotiation successful
0811 06:30:00.046111 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":716,"mutex_wait_us":120,"thread_start_us":412,"threads_started":1}
I20260811 06:30:00.048019 4494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c3fc01340094a50b4178a9954662d8c" candidate_uuid: "27607b5859494d26915b1ce2e2c22774" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ab6a4d25dbb46378021701be3bb0fd6" is_pre_election: true
I20260811 06:30:00.048024 4278 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c3fc01340094a50b4178a9954662d8c" candidate_uuid: "27607b5859494d26915b1ce2e2c22774" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3f60d945e1a4ea78490908fd6a4f8a0" is_pre_election: true
I20260811 06:30:00.048556 4494 raft_consensus.cc:2454] T 0c3fc01340094a50b4178a9954662d8c P 5ab6a4d25dbb46378021701be3bb0fd6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27607b5859494d26915b1ce2e2c22774 in term 0.
I20260811 06:30:00.048578 4278 raft_consensus.cc:2454] T 0c3fc01340094a50b4178a9954662d8c P c3f60d945e1a4ea78490908fd6a4f8a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27607b5859494d26915b1ce2e2c22774 in term 0.
I20260811 06:30:00.049299 3965 leader_election.cc:302] T 0c3fc01340094a50b4178a9954662d8c P 27607b5859494d26915b1ce2e2c22774 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27607b5859494d26915b1ce2e2c22774, 5ab6a4d25dbb46378021701be3bb0fd6; no voters:
I20260811 06:30:00.049795 4615 raft_consensus.cc:2781] T 0c3fc01340094a50b4178a9954662d8c P 27607b5859494d26915b1ce2e2c22774 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:30:00.049974 4615 raft_consensus.cc:482] T 0c3fc01340094a50b4178a9954662d8c P 27607b5859494d26915b1ce2e2c22774 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:30:00.050096 4615 raft_consensus.cc:3037] T 0c3fc01340094a50b4178a9954662d8c P 27607b5859494d26915b1ce2e2c22774 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:30:00.052320 4615 raft_consensus.cc:504] T 0c3fc01340094a50b4178a9954662d8c P 27607b5859494d26915b1ce2e2c22774 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3f60d945e1a4ea78490908fd6a4f8a0" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 35731 } } peers { permanent_uuid: "5ab6a4d25dbb46378021701be3bb0fd6" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 46713 } } peers { permanent_uuid: "27607b5859494d26915b1ce2e2c22774" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 33079 } }
I20260811 06:30:00.052943 4615 leader_election.cc:288] T 0c3fc01340094a50b4178a9954662d8c P 27607b5859494d26915b1ce2e2c22774 [CANDIDATE]: Term 1 election: Requested vote from peers c3f60d945e1a4ea78490908fd6a4f8a0 (127.28.239.194:35731), 5ab6a4d25dbb46378021701be3bb0fd6 (127.28.239.195:46713)
I20260811 06:30:00.053468 4278 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c3fc01340094a50b4178a9954662d8c" candidate_uuid: "27607b5859494d26915b1ce2e2c22774" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3f60d945e1a4ea78490908fd6a4f8a0"
I20260811 06:30:00.053494 4494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c3fc01340094a50b4178a9954662d8c" candidate_uuid: "27607b5859494d26915b1ce2e2c22774" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ab6a4d25dbb46378021701be3bb0fd6"
I20260811 06:30:00.053736 4278 raft_consensus.cc:3037] T 0c3fc01340094a50b4178a9954662d8c P c3f60d945e1a4ea78490908fd6a4f8a0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:30:00.053736 4494 raft_consensus.cc:3037] T 0c3fc01340094a50b4178a9954662d8c P 5ab6a4d25dbb46378021701be3bb0fd6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:30:00.057160 4278 raft_consensus.cc:2454] T 0c3fc01340094a50b4178a9954662d8c P c3f60d945e1a4ea78490908fd6a4f8a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27607b5859494d26915b1ce2e2c22774 in term 1.
I20260811 06:30:00.057160 4494 raft_consensus.cc:2454] T 0c3fc01340094a50b4178a9954662d8c P 5ab6a4d25dbb46378021701be3bb0fd6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27607b5859494d26915b1ce2e2c22774 in term 1.
I20260811 06:30:00.057725 3965 leader_election.cc:302] T 0c3fc01340094a50b4178a9954662d8c P 27607b5859494d26915b1ce2e2c22774 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27607b5859494d26915b1ce2e2c22774, 5ab6a4d25dbb46378021701be3bb0fd6; no voters:
I20260811 06:30:00.058122 4615 raft_consensus.cc:2781] T 0c3fc01340094a50b4178a9954662d8c P 27607b5859494d26915b1ce2e2c22774 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:30:00.058755 4615 raft_consensus.cc:686] T 0c3fc01340094a50b4178a9954662d8c P 27607b5859494d26915b1ce2e2c22774 [term 1 LEADER]: Becoming Leader. State: Replica: 27607b5859494d26915b1ce2e2c22774, State: Running, Role: LEADER
I20260811 06:30:00.059345 4615 consensus_queue.cc:237] T 0c3fc01340094a50b4178a9954662d8c P 27607b5859494d26915b1ce2e2c22774 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3f60d945e1a4ea78490908fd6a4f8a0" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 35731 } } peers { permanent_uuid: "5ab6a4d25dbb46378021701be3bb0fd6" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 46713 } } peers { permanent_uuid: "27607b5859494d26915b1ce2e2c22774" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 33079 } }
I20260811 06:30:00.066283 3848 catalog_manager.cc:5495] T 0c3fc01340094a50b4178a9954662d8c P 27607b5859494d26915b1ce2e2c22774 reported cstate change: term changed from 0 to 1, leader changed from <none> to 27607b5859494d26915b1ce2e2c22774 (127.28.239.193). New cstate: current_term: 1 leader_uuid: "27607b5859494d26915b1ce2e2c22774" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3f60d945e1a4ea78490908fd6a4f8a0" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 35731 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ab6a4d25dbb46378021701be3bb0fd6" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 46713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27607b5859494d26915b1ce2e2c22774" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 33079 } health_report { overall_health: HEALTHY } } }
W20260811 06:30:00.082886 4574 tablet.cc:2369] T 0c3fc01340094a50b4178a9954662d8c P 5ab6a4d25dbb46378021701be3bb0fd6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:30:00.109490 4604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:00.104163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36266 (local address 127.28.239.193:33079)
0811 06:30:00.104372 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:00.104377 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:00.104397 (+ 20us) server_negotiation.cc:407] Connection header received
0811 06:30:00.104491 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:00.104495 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:00.104578 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:00.104709 (+ 131us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:30:00.105052 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:00.105860 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:00.107209 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:00.107699 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:00.107801 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:00.108308 (+ 507us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:30:00.108896 (+ 588us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:30:00.109015 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:00.109118 (+ 103us) server_negotiation.cc:299] Negotiation successful
0811 06:30:00.109186 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 06:30:00.118317 4278 raft_consensus.cc:1261] T 0c3fc01340094a50b4178a9954662d8c P c3f60d945e1a4ea78490908fd6a4f8a0 [term 1 FOLLOWER]: Refusing update from remote peer 27607b5859494d26915b1ce2e2c22774: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:30:00.118420 4494 raft_consensus.cc:1261] T 0c3fc01340094a50b4178a9954662d8c P 5ab6a4d25dbb46378021701be3bb0fd6 [term 1 FOLLOWER]: Refusing update from remote peer 27607b5859494d26915b1ce2e2c22774: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:30:00.119423 4620 consensus_queue.cc:1035] T 0c3fc01340094a50b4178a9954662d8c P 27607b5859494d26915b1ce2e2c22774 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3f60d945e1a4ea78490908fd6a4f8a0" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 35731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:30:00.119776 4615 consensus_queue.cc:1035] T 0c3fc01340094a50b4178a9954662d8c P 27607b5859494d26915b1ce2e2c22774 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ab6a4d25dbb46378021701be3bb0fd6" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 46713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:30:00.131778 4629 mvcc.cc:205] Tried to move back new op lower bound from 7317216461273722880 to 7317216461047709696. Current Snapshot: MvccSnapshot[applied={T|T < 7317216461273722880}]
I20260811 06:30:00.132844 4633 mvcc.cc:205] Tried to move back new op lower bound from 7317216461273722880 to 7317216461047709696. Current Snapshot: MvccSnapshot[applied={T|T < 7317216461273722880}]
I20260811 06:30:00.147648 3848 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37062:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:30:00.147964 3848 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37062:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:30:00.151827 3848 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e95a4258059c432a9535729685e1f15e: Sending DeleteTablet for 3 replicas of tablet 0c3fc01340094a50b4178a9954662d8c
I20260811 06:30:00.153013 4246 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c3fc01340094a50b4178a9954662d8c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:30:00 UTC) from {username='kudu', principal='kudu/127.28.239.254@KRBTEST.COM'} at 127.0.0.1:60908
I20260811 06:30:00.153223 4452 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c3fc01340094a50b4178a9954662d8c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:30:00 UTC) from {username='kudu', principal='kudu/127.28.239.254@KRBTEST.COM'} at 127.0.0.1:54712
I20260811 06:30:00.154269 4035 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c3fc01340094a50b4178a9954662d8c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:30:00 UTC) from {username='kudu', principal='kudu/127.28.239.254@KRBTEST.COM'} at 127.0.0.1:36264
I20260811 06:30:00.155160 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 3939
I20260811 06:30:00.160727 4646 tablet_replica.cc:330] T 0c3fc01340094a50b4178a9954662d8c P 27607b5859494d26915b1ce2e2c22774: stopping tablet replica
I20260811 06:30:00.161280 4646 raft_consensus.cc:2229] T 0c3fc01340094a50b4178a9954662d8c P 27607b5859494d26915b1ce2e2c22774 [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:30:00.162146 4646 raft_consensus.cc:2258] T 0c3fc01340094a50b4178a9954662d8c P 27607b5859494d26915b1ce2e2c22774 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:30:00.169039 4644 tablet_replica.cc:330] T 0c3fc01340094a50b4178a9954662d8c P c3f60d945e1a4ea78490908fd6a4f8a0: stopping tablet replica
I20260811 06:30:00.169263 4646 ts_tablet_manager.cc:1905] T 0c3fc01340094a50b4178a9954662d8c P 27607b5859494d26915b1ce2e2c22774: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:30:00.169806 4644 raft_consensus.cc:2229] T 0c3fc01340094a50b4178a9954662d8c P c3f60d945e1a4ea78490908fd6a4f8a0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:30:00.170466 4644 raft_consensus.cc:2258] T 0c3fc01340094a50b4178a9954662d8c P c3f60d945e1a4ea78490908fd6a4f8a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:30:00.172886 4644 ts_tablet_manager.cc:1905] T 0c3fc01340094a50b4178a9954662d8c P c3f60d945e1a4ea78490908fd6a4f8a0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:30:00.174299 4645 tablet_replica.cc:330] T 0c3fc01340094a50b4178a9954662d8c P 5ab6a4d25dbb46378021701be3bb0fd6: stopping tablet replica
I20260811 06:30:00.174760 4645 raft_consensus.cc:2229] T 0c3fc01340094a50b4178a9954662d8c P 5ab6a4d25dbb46378021701be3bb0fd6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:30:00.175302 4645 raft_consensus.cc:2258] T 0c3fc01340094a50b4178a9954662d8c P 5ab6a4d25dbb46378021701be3bb0fd6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:30:00.177502 4645 ts_tablet_manager.cc:1905] T 0c3fc01340094a50b4178a9954662d8c P 5ab6a4d25dbb46378021701be3bb0fd6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:30:00.179368 4646 ts_tablet_manager.cc:1918] T 0c3fc01340094a50b4178a9954662d8c P 27607b5859494d26915b1ce2e2c22774: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:30:00.179548 4646 log.cc:1196] T 0c3fc01340094a50b4178a9954662d8c P 27607b5859494d26915b1ce2e2c22774: Deleting WAL directory at /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/minicluster-data/ts-0/wal/wals/0c3fc01340094a50b4178a9954662d8c
I20260811 06:30:00.180150 4646 ts_tablet_manager.cc:1939] T 0c3fc01340094a50b4178a9954662d8c P 27607b5859494d26915b1ce2e2c22774: Deleting consensus metadata
I20260811 06:30:00.180802 4644 ts_tablet_manager.cc:1918] T 0c3fc01340094a50b4178a9954662d8c P c3f60d945e1a4ea78490908fd6a4f8a0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:30:00.181056 4644 log.cc:1196] T 0c3fc01340094a50b4178a9954662d8c P c3f60d945e1a4ea78490908fd6a4f8a0: Deleting WAL directory at /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/minicluster-data/ts-1/wal/wals/0c3fc01340094a50b4178a9954662d8c
I20260811 06:30:00.181756 4644 ts_tablet_manager.cc:1939] T 0c3fc01340094a50b4178a9954662d8c P c3f60d945e1a4ea78490908fd6a4f8a0: Deleting consensus metadata
I20260811 06:30:00.183866 3828 catalog_manager.cc:4841] TS c3f60d945e1a4ea78490908fd6a4f8a0 (127.28.239.194:35731): tablet 0c3fc01340094a50b4178a9954662d8c (table test-table [id=e29671a0ab304d41a78539596ecabeda]) successfully deleted
I20260811 06:30:00.184710 4645 ts_tablet_manager.cc:1918] T 0c3fc01340094a50b4178a9954662d8c P 5ab6a4d25dbb46378021701be3bb0fd6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:30:00.184911 4645 log.cc:1196] T 0c3fc01340094a50b4178a9954662d8c P 5ab6a4d25dbb46378021701be3bb0fd6: Deleting WAL directory at /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786429755927508-29631-0/minicluster-data/ts-2/wal/wals/0c3fc01340094a50b4178a9954662d8c
I20260811 06:30:00.184899 3829 catalog_manager.cc:4841] TS 27607b5859494d26915b1ce2e2c22774 (127.28.239.193:33079): tablet 0c3fc01340094a50b4178a9954662d8c (table test-table [id=e29671a0ab304d41a78539596ecabeda]) successfully deleted
Aug 11 06:30:00 dist-test-slave-gkw7 krb5kdc[3779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429797, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.239.193@KRBTEST.COM
I20260811 06:30:00.185448 4645 ts_tablet_manager.cc:1939] T 0c3fc01340094a50b4178a9954662d8c P 5ab6a4d25dbb46378021701be3bb0fd6: Deleting consensus metadata
I20260811 06:30:00.187208 3829 catalog_manager.cc:4841] TS 5ab6a4d25dbb46378021701be3bb0fd6 (127.28.239.195:46713): tablet 0c3fc01340094a50b4178a9954662d8c (table test-table [id=e29671a0ab304d41a78539596ecabeda]) successfully deleted
I20260811 06:30:00.192818 4604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:00.156435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36274 (local address 127.28.239.193:33079)
0811 06:30:00.156619 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:00.156625 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:00.157665 (+ 1040us) server_negotiation.cc:407] Connection header received
0811 06:30:00.157954 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:00.157963 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:00.158066 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:00.158156 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:00.162643 (+ 4487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:00.164298 (+ 1655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:00.171591 (+ 7293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:00.172304 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:00.172423 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:00.186393 (+ 13970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:00.186405 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:00.186420 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:30:00.186462 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:00.189835 (+ 3373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:00.190574 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:00.190587 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:00.190590 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:00.190757 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:00.191174 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:00.191178 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:00.191180 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:00.191552 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:00.191876 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:00.192193 (+ 317us) server_negotiation.cc:299] Negotiation successful
0811 06:30:00.192450 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 06:30:00.193569 4128 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:30:00.295578 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 3939
I20260811 06:30:00.325518 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 4148
Aug 11 06:30:00 dist-test-slave-gkw7 krb5kdc[3779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429797, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.239.194@KRBTEST.COM
I20260811 06:30:00.340713 4597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:00.326530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60914 (local address 127.28.239.194:35731)
0811 06:30:00.326721 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:00.326726 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:00.326797 (+ 71us) server_negotiation.cc:407] Connection header received
0811 06:30:00.326983 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:00.326987 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:00.327064 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:00.327156 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:00.328388 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:00.329141 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:00.330405 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:00.330796 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:00.330892 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:00.335025 (+ 4133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:00.335040 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:00.335057 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:30:00.335096 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:00.338209 (+ 3113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:00.338832 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:00.338840 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:00.338842 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:00.338966 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:00.339292 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:00.339296 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:00.339298 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:00.339675 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:00.339902 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:00.340146 (+ 244us) server_negotiation.cc:299] Negotiation successful
0811 06:30:00.340332 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 06:30:00.341459 4338 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:30:00.434813 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 4148
I20260811 06:30:00.461508 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 4359
Aug 11 06:30:00 dist-test-slave-gkw7 krb5kdc[3779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429797, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.239.195@KRBTEST.COM
I20260811 06:30:00.477258 4596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:00.462400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54716 (local address 127.28.239.195:46713)
0811 06:30:00.462571 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:00.462576 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:00.462617 (+ 41us) server_negotiation.cc:407] Connection header received
0811 06:30:00.462786 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:00.462791 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:00.462858 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:00.462946 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:00.464253 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:00.464982 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:00.466416 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:00.466814 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:00.466924 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:00.471518 (+ 4594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:00.471536 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:00.471553 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:30:00.471597 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:00.474617 (+ 3020us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:00.475246 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:00.475254 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:00.475256 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:00.475387 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:00.475805 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:00.475819 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:00.475822 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:00.476269 (+ 447us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:00.476529 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:00.476759 (+ 230us) server_negotiation.cc:299] Negotiation successful
0811 06:30:00.476933 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 06:30:00.478045 4568 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:30:00.570462 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 4359
I20260811 06:30:00.596475 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 3803
I20260811 06:30:00.597698 3894 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:30:00.710587 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 3803
2026-08-11T06:30:00Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4696 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-11T06:30:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:30:00Z Disabled control of system clock
I20260811 06:30:00.799927 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429755927508-29631-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429755927508-29631-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429755927508-29631-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:33401
--webserver_interface=127.28.239.254
--webserver_port=0
--builtin_ntp_servers=127.28.239.212:35481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.239.254:33401
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 06:30:01.147331 4661 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:30:01.147635 4661 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:30:01.147722 4661 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:30:01.155551 4661 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:30:01.155668 4661 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:30:01.155706 4661 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:30:01.155757 4661 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:30:01.155809 4661 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:30:01.165299 4661 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:35481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429755927508-29631-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429755927508-29631-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.239.254:33401
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:33401
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.239.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429755927508-29631-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:30:01.166723 4661 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:30:01.168535 4661 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:30:01.181102 4674 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:30:01.181174 4667 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:30:01.181202 4669 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:30:01.182260 4661 server_base.cc:1034] running on GCE node
I20260811 06:30:01.183116 4661 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:30:01.184628 4661 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:30:01.185868 4661 hybrid_clock.cc:648] HybridClock initialized: now 1786429801185830 us; error 60 us; skew 500 ppm
I20260811 06:30:01.186283 4661 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:30:01.188680 4661 webserver.cc:460] Webserver started at http://127.28.239.254:32847/ using document root <none> and password file <none>
I20260811 06:30:01.189441 4661 fs_manager.cc:359] Metadata directory not provided
I20260811 06:30:01.189596 4661 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:30:01.189944 4661 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:30:01.192975 4661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429755927508-29631-0/minicluster-data/master-0/data/instance:
uuid: "1db3b579ecf949a38d2d108ae42a9b53"
format_stamp: "Formatted at 2026-08-11 06:30:01 on dist-test-slave-gkw7"
server_key: "6caed54726aaa0403fe6a3aa62b0c171"
server_key_iv: "c10917f6f5f48e0729797996ea93b6fd"
server_key_version: "encryptionkey@0"
I20260811 06:30:01.193886 4661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429755927508-29631-0/minicluster-data/master-0/wal/instance:
uuid: "1db3b579ecf949a38d2d108ae42a9b53"
format_stamp: "Formatted at 2026-08-11 06:30:01 on dist-test-slave-gkw7"
server_key: "6caed54726aaa0403fe6a3aa62b0c171"
server_key_iv: "c10917f6f5f48e0729797996ea93b6fd"
server_key_version: "encryptionkey@0"
I20260811 06:30:01.199263 4661 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 06:30:01.202773 4683 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:01.204129 4661 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 06:30:01.204344 4661 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429755927508-29631-0/minicluster-data/master-0/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429755927508-29631-0/minicluster-data/master-0/wal
uuid: "1db3b579ecf949a38d2d108ae42a9b53"
format_stamp: "Formatted at 2026-08-11 06:30:01 on dist-test-slave-gkw7"
server_key: "6caed54726aaa0403fe6a3aa62b0c171"
server_key_iv: "c10917f6f5f48e0729797996ea93b6fd"
server_key_version: "encryptionkey@0"
I20260811 06:30:01.204583 4661 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429755927508-29631-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429755927508-29631-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429755927508-29631-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:30:01.210492 4661 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:30:01.211308 4661 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:30:01.211621 4661 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:30:01.230077 4661 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.254:33401
I20260811 06:30:01.230111 4766 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.254:33401 every 8 connection(s)
I20260811 06:30:01.231572 4661 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
I20260811 06:30:01.234215 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 4661
I20260811 06:30:01.234431 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429755927508-29631-0/minicluster-data/master-0/wal/instance
I20260811 06:30:01.235287 29631 external_mini_cluster.cc:1428] Setting key 4684ff6d0c808a6a15cc8980489aeb5b
I20260811 06:30:01.236341 4767 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:30:01.245074 4772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:01.237630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59330 (local address 127.28.239.254:33401)
0811 06:30:01.238711 (+ 1081us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:01.238723 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:01.238766 (+ 43us) server_negotiation.cc:407] Connection header received
0811 06:30:01.241190 (+ 2424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:01.241213 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:01.241723 (+ 510us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:01.242088 (+ 365us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:01.243336 (+ 1248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:01.243343 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:01.243360 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:30:01.243375 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:01.243445 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:30:01.243465 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:01.243639 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:01.243807 (+ 168us) server_negotiation.cc:299] Negotiation successful
0811 06:30:01.244098 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":623,"thread_start_us":425,"threads_started":1}
I20260811 06:30:01.250746 4767 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1db3b579ecf949a38d2d108ae42a9b53: Bootstrap starting.
I20260811 06:30:01.254231 4767 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1db3b579ecf949a38d2d108ae42a9b53: Neither blocks nor log segments found. Creating new log.
I20260811 06:30:01.255410 4767 log.cc:824] T 00000000000000000000000000000000 P 1db3b579ecf949a38d2d108ae42a9b53: Log is configured to *not* fsync() on all Append() calls
I20260811 06:30:01.258636 4767 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1db3b579ecf949a38d2d108ae42a9b53: No bootstrap required, opened a new log
I20260811 06:30:01.266212 4767 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1db3b579ecf949a38d2d108ae42a9b53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1db3b579ecf949a38d2d108ae42a9b53" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 33401 } }
I20260811 06:30:01.266649 4767 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1db3b579ecf949a38d2d108ae42a9b53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:30:01.266763 4767 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1db3b579ecf949a38d2d108ae42a9b53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1db3b579ecf949a38d2d108ae42a9b53, State: Initialized, Role: FOLLOWER
I20260811 06:30:01.267362 4767 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1db3b579ecf949a38d2d108ae42a9b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1db3b579ecf949a38d2d108ae42a9b53" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 33401 } }
I20260811 06:30:01.267627 4767 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1db3b579ecf949a38d2d108ae42a9b53 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:30:01.267778 4767 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1db3b579ecf949a38d2d108ae42a9b53 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:30:01.267964 4767 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1db3b579ecf949a38d2d108ae42a9b53 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:30:01.270323 4767 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1db3b579ecf949a38d2d108ae42a9b53 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1db3b579ecf949a38d2d108ae42a9b53" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 33401 } }
I20260811 06:30:01.270866 4767 leader_election.cc:302] T 00000000000000000000000000000000 P 1db3b579ecf949a38d2d108ae42a9b53 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1db3b579ecf949a38d2d108ae42a9b53; no voters:
I20260811 06:30:01.271551 4767 leader_election.cc:288] T 00000000000000000000000000000000 P 1db3b579ecf949a38d2d108ae42a9b53 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:30:01.271700 4775 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1db3b579ecf949a38d2d108ae42a9b53 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:30:01.274339 4767 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1db3b579ecf949a38d2d108ae42a9b53 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:30:01.278177 4775 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1db3b579ecf949a38d2d108ae42a9b53 [term 1 LEADER]: Becoming Leader. State: Replica: 1db3b579ecf949a38d2d108ae42a9b53, State: Running, Role: LEADER
I20260811 06:30:01.278954 4775 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1db3b579ecf949a38d2d108ae42a9b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1db3b579ecf949a38d2d108ae42a9b53" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 33401 } }
I20260811 06:30:01.282414 4776 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1db3b579ecf949a38d2d108ae42a9b53 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1db3b579ecf949a38d2d108ae42a9b53" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1db3b579ecf949a38d2d108ae42a9b53" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 33401 } } }
I20260811 06:30:01.282819 4776 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1db3b579ecf949a38d2d108ae42a9b53 [sys.catalog]: This master's current role is: LEADER
I20260811 06:30:01.283350 4777 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1db3b579ecf949a38d2d108ae42a9b53 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1db3b579ecf949a38d2d108ae42a9b53. Latest consensus state: current_term: 1 leader_uuid: "1db3b579ecf949a38d2d108ae42a9b53" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1db3b579ecf949a38d2d108ae42a9b53" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 33401 } } }
I20260811 06:30:01.283722 4777 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1db3b579ecf949a38d2d108ae42a9b53 [sys.catalog]: This master's current role is: LEADER
I20260811 06:30:01.286232 4790 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:30:01.290825 4790 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:30:01.299961 4790 catalog_manager.cc:1347] Generated new cluster ID: 190857881bf74503afe6454307916056
I20260811 06:30:01.300098 4790 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:30:01.313082 4790 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:30:01.313654 4790 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:30:01.324039 4790 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1db3b579ecf949a38d2d108ae42a9b53: Generated new TSK 0
I20260811 06:30:01.325136 4790 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:30:01.331907 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429755927508-29631-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429755927508-29631-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429755927508-29631-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.193:0
--local_ip_for_outbound_sockets=127.28.239.193
--webserver_interface=127.28.239.193
--webserver_port=0
--tserver_master_addrs=127.28.239.254:33401
--builtin_ntp_servers=127.28.239.212:35481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 06:30:01.702984 4801 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:30:01.703305 4801 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:30:01.703459 4801 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:30:01.711597 4801 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:30:01.711709 4801 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:30:01.711925 4801 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.193
I20260811 06:30:01.721266 4801 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:35481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429755927508-29631-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429755927508-29631-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.239.193
--webserver_port=0
--tserver_master_addrs=127.28.239.254:33401
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.193
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429755927508-29631-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:30:01.722738 4801 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:30:01.725082 4801 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:30:01.739317 4809 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:30:01.739482 4811 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:30:01.739434 4807 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:30:01.740679 4801 server_base.cc:1034] running on GCE node
I20260811 06:30:01.741376 4801 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:30:01.742466 4801 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:30:01.743717 4801 hybrid_clock.cc:648] HybridClock initialized: now 1786429801743664 us; error 44 us; skew 500 ppm
I20260811 06:30:01.744093 4801 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:30:01.746738 4801 webserver.cc:460] Webserver started at http://127.28.239.193:46393/ using document root <none> and password file <none>
I20260811 06:30:01.747475 4801 fs_manager.cc:359] Metadata directory not provided
I20260811 06:30:01.747574 4801 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:30:01.747874 4801 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:30:01.750982 4801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429755927508-29631-0/minicluster-data/ts-0/data/instance:
uuid: "7dc50b1b1f0b4075bfca22384b64e2e3"
format_stamp: "Formatted at 2026-08-11 06:30:01 on dist-test-slave-gkw7"
server_key: "00bbf74697b5c7b211e4c7591c0c4e16"
server_key_iv: "1e42e48552c2ed14aa3fd80d8edad2fb"
server_key_version: "encryptionkey@0"
I20260811 06:30:01.751787 4801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429755927508-29631-0/minicluster-data/ts-0/wal/instance:
uuid: "7dc50b1b1f0b4075bfca22384b64e2e3"
format_stamp: "Formatted at 2026-08-11 06:30:01 on dist-test-slave-gkw7"
server_key: "00bbf74697b5c7b211e4c7591c0c4e16"
server_key_iv: "1e42e48552c2ed14aa3fd80d8edad2fb"
server_key_version: "encryptionkey@0"
I20260811 06:30:01.757476 4801 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260811 06:30:01.761273 4820 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:01.762861 4801 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:30:01.763016 4801 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429755927508-29631-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429755927508-29631-0/minicluster-data/ts-0/wal
uuid: "7dc50b1b1f0b4075bfca22384b64e2e3"
format_stamp: "Formatted at 2026-08-11 06:30:01 on dist-test-slave-gkw7"
server_key: "00bbf74697b5c7b211e4c7591c0c4e16"
server_key_iv: "1e42e48552c2ed14aa3fd80d8edad2fb"
server_key_version: "encryptionkey@0"
I20260811 06:30:01.763234 4801 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429755927508-29631-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429755927508-29631-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429755927508-29631-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:30:01.769500 4801 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:30:01.770421 4801 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:30:01.770671 4801 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:30:01.771615 4801 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:30:01.773061 4801 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:30:01.773159 4801 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:01.773257 4801 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:30:01.773308 4801 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:01.809827 4801 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.193:33199
I20260811 06:30:01.809947 4990 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.193:33199 every 8 connection(s)
I20260811 06:30:01.811308 4801 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
I20260811 06:30:01.818312 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 4801
I20260811 06:30:01.818538 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429755927508-29631-0/minicluster-data/ts-0/wal/instance
I20260811 06:30:01.819206 29631 external_mini_cluster.cc:1428] Setting key 2a91dd6cbd9fed983bceed733626643c
I20260811 06:30:01.820678 4994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:01.814302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.193:44937 (local address 127.28.239.254:33401)
0811 06:30:01.814779 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:01.814784 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:01.816168 (+ 1384us) server_negotiation.cc:407] Connection header received
0811 06:30:01.818031 (+ 1863us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:01.818036 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:01.818130 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:01.818257 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:01.819810 (+ 1553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:01.819816 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:01.819821 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:30:01.819843 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:01.819892 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:30:01.819904 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:01.820046 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:01.820248 (+ 202us) server_negotiation.cc:299] Negotiation successful
0811 06:30:01.820322 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":244,"threads_started":1}
I20260811 06:30:01.821756 4996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:01.814954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:33401 (local address 127.28.239.193:44937)
0811 06:30:01.815950 (+ 996us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:01.815988 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:01.817718 (+ 1730us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:01.818553 (+ 835us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:01.818563 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:01.818713 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:01.819487 (+ 774us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 06:30:01.819490 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:01.819530 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 06:30:01.819630 (+ 100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:30:01.820003 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:30:01.820016 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:30:01.820027 (+ 11us) client_negotiation.cc:769] Sending connection context
0811 06:30:01.820341 (+ 314us) client_negotiation.cc:240] Negotiation successful
0811 06:30:01.820646 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":302,"threads_started":1}
I20260811 06:30:01.822355 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429755927508-29631-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429755927508-29631-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429755927508-29631-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.194:0
--local_ip_for_outbound_sockets=127.28.239.194
--webserver_interface=127.28.239.194
--webserver_port=0
--tserver_master_addrs=127.28.239.254:33401
--builtin_ntp_servers=127.28.239.212:35481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260811 06:30:01.823413 4991 heartbeater.cc:347] Connected to a master server at 127.28.239.254:33401
I20260811 06:30:01.823802 4991 heartbeater.cc:464] Registering TS with master...
I20260811 06:30:01.824493 4991 heartbeater.cc:511] Master 127.28.239.254:33401 requested a full tablet report, sending...
I20260811 06:30:01.826748 4711 ts_manager.cc:194] Registered new tserver with Master: 7dc50b1b1f0b4075bfca22384b64e2e3 (127.28.239.193:33199)
W20260811 06:30:02.178380 4997 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:30:02.178699 4997 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:30:02.178864 4997 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:30:02.186446 4997 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:30:02.186571 4997 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:30:02.186764 4997 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.194
I20260811 06:30:02.196271 4997 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:35481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429755927508-29631-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429755927508-29631-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.239.194
--webserver_port=0
--tserver_master_addrs=127.28.239.254:33401
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.194
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429755927508-29631-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:30:02.197723 4997 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:30:02.199642 4997 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:30:02.213984 5005 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:30:02.214074 5010 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:30:02.214010 5007 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:30:02.214990 4997 server_base.cc:1034] running on GCE node
I20260811 06:30:02.215613 4997 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:30:02.216641 4997 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:30:02.217880 4997 hybrid_clock.cc:648] HybridClock initialized: now 1786429802217820 us; error 50 us; skew 500 ppm
I20260811 06:30:02.218236 4997 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:30:02.220659 4997 webserver.cc:460] Webserver started at http://127.28.239.194:45661/ using document root <none> and password file <none>
I20260811 06:30:02.221431 4997 fs_manager.cc:359] Metadata directory not provided
I20260811 06:30:02.221594 4997 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:30:02.221987 4997 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:30:02.225133 4997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429755927508-29631-0/minicluster-data/ts-1/data/instance:
uuid: "b5ad989a5dfa480db4819077922f1f75"
format_stamp: "Formatted at 2026-08-11 06:30:02 on dist-test-slave-gkw7"
server_key: "d4b63b52702afe664afb72c4d654ff15"
server_key_iv: "5d81dd24852135a1cef163878fe88867"
server_key_version: "encryptionkey@0"
I20260811 06:30:02.226068 4997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429755927508-29631-0/minicluster-data/ts-1/wal/instance:
uuid: "b5ad989a5dfa480db4819077922f1f75"
format_stamp: "Formatted at 2026-08-11 06:30:02 on dist-test-slave-gkw7"
server_key: "d4b63b52702afe664afb72c4d654ff15"
server_key_iv: "5d81dd24852135a1cef163878fe88867"
server_key_version: "encryptionkey@0"
I20260811 06:30:02.232391 4997 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.004s
I20260811 06:30:02.237217 5018 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:02.239164 4997 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 06:30:02.239357 4997 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429755927508-29631-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429755927508-29631-0/minicluster-data/ts-1/wal
uuid: "b5ad989a5dfa480db4819077922f1f75"
format_stamp: "Formatted at 2026-08-11 06:30:02 on dist-test-slave-gkw7"
server_key: "d4b63b52702afe664afb72c4d654ff15"
server_key_iv: "5d81dd24852135a1cef163878fe88867"
server_key_version: "encryptionkey@0"
I20260811 06:30:02.239614 4997 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429755927508-29631-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429755927508-29631-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429755927508-29631-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:30:02.247635 4997 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:30:02.248682 4997 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:30:02.249042 4997 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:30:02.250233 4997 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:30:02.252278 4997 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:30:02.252370 4997 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:02.252466 4997 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:30:02.252524 4997 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:02.288408 4997 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.194:37585
I20260811 06:30:02.288461 5185 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.194:37585 every 8 connection(s)
I20260811 06:30:02.290074 4997 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
I20260811 06:30:02.299067 4994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:02.292865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.194:41975 (local address 127.28.239.254:33401)
0811 06:30:02.293057 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:02.293061 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:02.294687 (+ 1626us) server_negotiation.cc:407] Connection header received
0811 06:30:02.296502 (+ 1815us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:02.296509 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:02.296612 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:02.296733 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:02.298222 (+ 1489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:02.298238 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:02.298243 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:30:02.298271 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:02.298324 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:30:02.298336 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:02.298467 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:02.298675 (+ 208us) server_negotiation.cc:299] Negotiation successful
0811 06:30:02.298776 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 06:30:02.299194 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 4997
I20260811 06:30:02.299408 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429755927508-29631-0/minicluster-data/ts-1/wal/instance
I20260811 06:30:02.300076 29631 external_mini_cluster.cc:1428] Setting key fe9c11785a00d44c60d158eefc7ed53f
I20260811 06:30:02.300134 5189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:02.293446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:33401 (local address 127.28.239.194:41975)
0811 06:30:02.294513 (+ 1067us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:02.294551 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:02.296207 (+ 1656us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:02.296985 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:02.296995 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:02.297093 (+ 98us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:02.297893 (+ 800us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 06:30:02.297896 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:02.297944 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 06:30:02.298038 (+ 94us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:30:02.298437 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:30:02.298448 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:30:02.298456 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 06:30:02.298778 (+ 322us) client_negotiation.cc:240] Negotiation successful
0811 06:30:02.299061 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"thread_start_us":303,"threads_started":1}
I20260811 06:30:02.302054 5186 heartbeater.cc:347] Connected to a master server at 127.28.239.254:33401
I20260811 06:30:02.302381 5186 heartbeater.cc:464] Registering TS with master...
I20260811 06:30:02.303002 5186 heartbeater.cc:511] Master 127.28.239.254:33401 requested a full tablet report, sending...
I20260811 06:30:02.303617 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429755927508-29631-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429755927508-29631-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429755927508-29631-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.195:0
--local_ip_for_outbound_sockets=127.28.239.195
--webserver_interface=127.28.239.195
--webserver_port=0
--tserver_master_addrs=127.28.239.254:33401
--builtin_ntp_servers=127.28.239.212:35481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260811 06:30:02.304503 4710 ts_manager.cc:194] Registered new tserver with Master: b5ad989a5dfa480db4819077922f1f75 (127.28.239.194:37585)
W20260811 06:30:02.669298 5190 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:30:02.669623 5190 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:30:02.669718 5190 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:30:02.677136 5190 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:30:02.677255 5190 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:30:02.677371 5190 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.195
I20260811 06:30:02.686807 5190 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:35481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429755927508-29631-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429755927508-29631-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.239.195
--webserver_port=0
--tserver_master_addrs=127.28.239.254:33401
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.195
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429755927508-29631-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:30:02.688136 5190 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:30:02.689953 5190 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:30:02.703647 5197 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:30:02.703596 5204 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:30:02.703687 5199 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:30:02.704890 5190 server_base.cc:1034] running on GCE node
I20260811 06:30:02.705624 5190 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:30:02.706673 5190 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:30:02.707913 5190 hybrid_clock.cc:648] HybridClock initialized: now 1786429802707869 us; error 62 us; skew 500 ppm
I20260811 06:30:02.708297 5190 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:30:02.711023 5190 webserver.cc:460] Webserver started at http://127.28.239.195:40043/ using document root <none> and password file <none>
I20260811 06:30:02.711786 5190 fs_manager.cc:359] Metadata directory not provided
I20260811 06:30:02.711930 5190 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:30:02.712298 5190 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:30:02.715350 5190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429755927508-29631-0/minicluster-data/ts-2/data/instance:
uuid: "3f35a41c31614632b92d2d071a976173"
format_stamp: "Formatted at 2026-08-11 06:30:02 on dist-test-slave-gkw7"
server_key: "4e3f9c9b54a285b78b43c19f056b6792"
server_key_iv: "ea2d069fa2d3a9746d8f31d3bd207a2b"
server_key_version: "encryptionkey@0"
I20260811 06:30:02.716190 5190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429755927508-29631-0/minicluster-data/ts-2/wal/instance:
uuid: "3f35a41c31614632b92d2d071a976173"
format_stamp: "Formatted at 2026-08-11 06:30:02 on dist-test-slave-gkw7"
server_key: "4e3f9c9b54a285b78b43c19f056b6792"
server_key_iv: "ea2d069fa2d3a9746d8f31d3bd207a2b"
server_key_version: "encryptionkey@0"
I20260811 06:30:02.721596 5190 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260811 06:30:02.725337 5212 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:02.726811 5190 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 06:30:02.727010 5190 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429755927508-29631-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429755927508-29631-0/minicluster-data/ts-2/wal
uuid: "3f35a41c31614632b92d2d071a976173"
format_stamp: "Formatted at 2026-08-11 06:30:02 on dist-test-slave-gkw7"
server_key: "4e3f9c9b54a285b78b43c19f056b6792"
server_key_iv: "ea2d069fa2d3a9746d8f31d3bd207a2b"
server_key_version: "encryptionkey@0"
I20260811 06:30:02.727259 5190 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429755927508-29631-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429755927508-29631-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429755927508-29631-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:30:02.733297 5190 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:30:02.734171 5190 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:30:02.734467 5190 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:30:02.735411 5190 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:30:02.737041 5190 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:30:02.737169 5190 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:02.737311 5190 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:30:02.737403 5190 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:02.772719 5190 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.195:45275
I20260811 06:30:02.772787 5390 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.195:45275 every 8 connection(s)
I20260811 06:30:02.774417 5190 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
I20260811 06:30:02.779706 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 5190
I20260811 06:30:02.779973 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429755927508-29631-0/minicluster-data/ts-2/wal/instance
I20260811 06:30:02.780491 29631 external_mini_cluster.cc:1428] Setting key 6415b6b17e88af9da169ebb52f414db8
I20260811 06:30:02.784113 4994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:02.777306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.195:60561 (local address 127.28.239.254:33401)
0811 06:30:02.777564 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:02.777571 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:02.779385 (+ 1814us) server_negotiation.cc:407] Connection header received
0811 06:30:02.781568 (+ 2183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:02.781574 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:02.781651 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:02.781798 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:02.783376 (+ 1578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:02.783381 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:02.783386 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:30:02.783405 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:02.783446 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:30:02.783458 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:02.783580 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:02.783835 (+ 255us) server_negotiation.cc:299] Negotiation successful
0811 06:30:02.783888 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 06:30:02.785125 5399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:02.778017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:33401 (local address 127.28.239.195:60561)
0811 06:30:02.779202 (+ 1185us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:02.779257 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:02.781240 (+ 1983us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:02.782100 (+ 860us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:02.782112 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:02.782252 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:02.783060 (+ 808us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 06:30:02.783063 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:02.783104 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 06:30:02.783199 (+ 95us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:30:02.783562 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:30:02.783572 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:30:02.783580 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 06:30:02.783910 (+ 330us) client_negotiation.cc:240] Negotiation successful
0811 06:30:02.784147 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":611,"thread_start_us":408,"threads_started":1}
I20260811 06:30:02.786516 5392 heartbeater.cc:347] Connected to a master server at 127.28.239.254:33401
I20260811 06:30:02.786852 5392 heartbeater.cc:464] Registering TS with master...
I20260811 06:30:02.787446 5392 heartbeater.cc:511] Master 127.28.239.254:33401 requested a full tablet report, sending...
I20260811 06:30:02.788885 4710 ts_manager.cc:194] Registered new tserver with Master: 3f35a41c31614632b92d2d071a976173 (127.28.239.195:45275)
I20260811 06:30:02.796182 29631 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 06:30:02.803658 4994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:02.800800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59342 (local address 127.28.239.254:33401)
0811 06:30:02.801015 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:02.801024 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:02.801497 (+ 473us) server_negotiation.cc:407] Connection header received
0811 06:30:02.801768 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:02.801772 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:02.801840 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:02.801951 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:02.802965 (+ 1014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:02.802971 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:02.802976 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:30:02.802997 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:02.803047 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:30:02.803059 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:02.803198 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:02.803318 (+ 120us) server_negotiation.cc:299] Negotiation successful
0811 06:30:02.803383 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 06:30:02.807703 4710 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59342:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:30:02.815237 4710 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:30:02.829859 4991 heartbeater.cc:503] Master 127.28.239.254:33401 was elected leader, sending a full tablet report...
I20260811 06:30:02.835971 5417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:02.828393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60372 (local address 127.28.239.194:37585)
0811 06:30:02.829061 (+ 668us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:02.829071 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:02.829668 (+ 597us) server_negotiation.cc:407] Connection header received
0811 06:30:02.829946 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:02.829957 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:02.830231 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:02.830373 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:02.834324 (+ 3951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:02.834337 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:02.834355 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:30:02.834407 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:02.834516 (+ 109us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:30:02.834548 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:02.834787 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:02.835312 (+ 525us) server_negotiation.cc:299] Negotiation successful
0811 06:30:02.835464 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":500,"thread_start_us":341,"threads_started":1}
I20260811 06:30:02.836905 5419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:02.827742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.194:37585 (local address 127.0.0.1:60372)
0811 06:30:02.829518 (+ 1776us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:02.829559 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:02.829764 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:02.832306 (+ 2542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:02.832317 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:02.832450 (+ 133us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:02.833878 (+ 1428us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 06:30:02.833883 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:02.833916 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 06:30:02.834014 (+ 98us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:30:02.834963 (+ 949us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:30:02.834976 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:30:02.834983 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 06:30:02.836313 (+ 1330us) client_negotiation.cc:240] Negotiation successful
0811 06:30:02.836438 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1631,"threads_started":1}
I20260811 06:30:02.840981 5419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:02.828660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.193:33199 (local address 127.0.0.1:43208)
0811 06:30:02.837102 (+ 8442us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:02.837120 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:02.837198 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:02.838435 (+ 1237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:02.838439 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:02.838464 (+ 25us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:02.839094 (+ 630us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 06:30:02.839100 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:02.839119 (+ 19us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 06:30:02.839160 (+ 41us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:30:02.840549 (+ 1389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:30:02.840554 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:30:02.840560 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 06:30:02.840661 (+ 101us) client_negotiation.cc:240] Negotiation successful
0811 06:30:02.840713 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8346,"thread_start_us":907,"threads_started":1}
I20260811 06:30:02.841190 5420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:02.829306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43208 (local address 127.28.239.193:33199)
0811 06:30:02.837750 (+ 8444us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:02.837761 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:02.837827 (+ 66us) server_negotiation.cc:407] Connection header received
0811 06:30:02.837953 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:02.837963 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:02.838222 (+ 259us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:02.838783 (+ 561us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:02.840268 (+ 1485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:02.840278 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:02.840295 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:30:02.840314 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:02.840377 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:30:02.840404 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:02.840600 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:02.840678 (+ 78us) server_negotiation.cc:299] Negotiation successful
0811 06:30:02.840794 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8260,"thread_start_us":8247,"threads_started":1}
I20260811 06:30:02.842767 5413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:02.827357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.195:45275 (local address 127.0.0.1:37554)
0811 06:30:02.828265 (+ 908us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:02.828307 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:02.828451 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:02.839144 (+ 10693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:02.839148 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:02.839169 (+ 21us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:02.839828 (+ 659us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 06:30:02.839832 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:02.839851 (+ 19us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 06:30:02.839895 (+ 44us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:30:02.842175 (+ 2280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:30:02.842179 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:30:02.842186 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 06:30:02.842336 (+ 150us) client_negotiation.cc:240] Negotiation successful
0811 06:30:02.842392 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":659,"thread_start_us":487,"threads_started":1}
I20260811 06:30:02.842804 5415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:02.827588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37554 (local address 127.28.239.195:45275)
0811 06:30:02.838588 (+ 11000us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:02.838598 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:02.838635 (+ 37us) server_negotiation.cc:407] Connection header received
0811 06:30:02.838716 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:02.838725 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:02.838943 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:02.839562 (+ 619us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:02.841938 (+ 2376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:02.841947 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:02.841959 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:30:02.841976 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:02.842026 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:30:02.842044 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:02.842238 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:02.842308 (+ 70us) server_negotiation.cc:299] Negotiation successful
0811 06:30:02.842399 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10821,"thread_start_us":10624,"threads_started":1}
I20260811 06:30:02.842411 5088 tablet_service.cc:1467] Processing CreateTablet for tablet 75361ec40f3d46c493fbcf291acb97d4 (DEFAULT_TABLE table=test-table [id=9e276f2376244401bafdc63b26553d35]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:30:02.843849 5088 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75361ec40f3d46c493fbcf291acb97d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:30:02.845041 4895 tablet_service.cc:1467] Processing CreateTablet for tablet 75361ec40f3d46c493fbcf291acb97d4 (DEFAULT_TABLE table=test-table [id=9e276f2376244401bafdc63b26553d35]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:30:02.846197 5293 tablet_service.cc:1467] Processing CreateTablet for tablet 75361ec40f3d46c493fbcf291acb97d4 (DEFAULT_TABLE table=test-table [id=9e276f2376244401bafdc63b26553d35]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:30:02.846782 4895 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75361ec40f3d46c493fbcf291acb97d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:30:02.847579 5293 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75361ec40f3d46c493fbcf291acb97d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:30:02.858264 5421 tablet_bootstrap.cc:492] T 75361ec40f3d46c493fbcf291acb97d4 P 7dc50b1b1f0b4075bfca22384b64e2e3: Bootstrap starting.
I20260811 06:30:02.861182 5422 tablet_bootstrap.cc:492] T 75361ec40f3d46c493fbcf291acb97d4 P b5ad989a5dfa480db4819077922f1f75: Bootstrap starting.
I20260811 06:30:02.862581 5423 tablet_bootstrap.cc:492] T 75361ec40f3d46c493fbcf291acb97d4 P 3f35a41c31614632b92d2d071a976173: Bootstrap starting.
I20260811 06:30:02.862617 5421 tablet_bootstrap.cc:654] T 75361ec40f3d46c493fbcf291acb97d4 P 7dc50b1b1f0b4075bfca22384b64e2e3: Neither blocks nor log segments found. Creating new log.
I20260811 06:30:02.863992 5421 log.cc:824] T 75361ec40f3d46c493fbcf291acb97d4 P 7dc50b1b1f0b4075bfca22384b64e2e3: Log is configured to *not* fsync() on all Append() calls
I20260811 06:30:02.865073 5422 tablet_bootstrap.cc:654] T 75361ec40f3d46c493fbcf291acb97d4 P b5ad989a5dfa480db4819077922f1f75: Neither blocks nor log segments found. Creating new log.
I20260811 06:30:02.865808 5423 tablet_bootstrap.cc:654] T 75361ec40f3d46c493fbcf291acb97d4 P 3f35a41c31614632b92d2d071a976173: Neither blocks nor log segments found. Creating new log.
I20260811 06:30:02.866400 5422 log.cc:824] T 75361ec40f3d46c493fbcf291acb97d4 P b5ad989a5dfa480db4819077922f1f75: Log is configured to *not* fsync() on all Append() calls
I20260811 06:30:02.866876 5423 log.cc:824] T 75361ec40f3d46c493fbcf291acb97d4 P 3f35a41c31614632b92d2d071a976173: Log is configured to *not* fsync() on all Append() calls
I20260811 06:30:02.867722 5421 tablet_bootstrap.cc:492] T 75361ec40f3d46c493fbcf291acb97d4 P 7dc50b1b1f0b4075bfca22384b64e2e3: No bootstrap required, opened a new log
I20260811 06:30:02.868141 5421 ts_tablet_manager.cc:1397] T 75361ec40f3d46c493fbcf291acb97d4 P 7dc50b1b1f0b4075bfca22384b64e2e3: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.006s
I20260811 06:30:02.870235 5423 tablet_bootstrap.cc:492] T 75361ec40f3d46c493fbcf291acb97d4 P 3f35a41c31614632b92d2d071a976173: No bootstrap required, opened a new log
I20260811 06:30:02.870239 5422 tablet_bootstrap.cc:492] T 75361ec40f3d46c493fbcf291acb97d4 P b5ad989a5dfa480db4819077922f1f75: No bootstrap required, opened a new log
I20260811 06:30:02.870507 5423 ts_tablet_manager.cc:1397] T 75361ec40f3d46c493fbcf291acb97d4 P 3f35a41c31614632b92d2d071a976173: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260811 06:30:02.870507 5422 ts_tablet_manager.cc:1397] T 75361ec40f3d46c493fbcf291acb97d4 P b5ad989a5dfa480db4819077922f1f75: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260811 06:30:02.875861 5421 raft_consensus.cc:348] T 75361ec40f3d46c493fbcf291acb97d4 P 7dc50b1b1f0b4075bfca22384b64e2e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f35a41c31614632b92d2d071a976173" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 45275 } } peers { permanent_uuid: "7dc50b1b1f0b4075bfca22384b64e2e3" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 33199 } } peers { permanent_uuid: "b5ad989a5dfa480db4819077922f1f75" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37585 } }
I20260811 06:30:02.876261 5421 raft_consensus.cc:374] T 75361ec40f3d46c493fbcf291acb97d4 P 7dc50b1b1f0b4075bfca22384b64e2e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:30:02.876375 5421 raft_consensus.cc:729] T 75361ec40f3d46c493fbcf291acb97d4 P 7dc50b1b1f0b4075bfca22384b64e2e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dc50b1b1f0b4075bfca22384b64e2e3, State: Initialized, Role: FOLLOWER
I20260811 06:30:02.877023 5421 consensus_queue.cc:260] T 75361ec40f3d46c493fbcf291acb97d4 P 7dc50b1b1f0b4075bfca22384b64e2e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f35a41c31614632b92d2d071a976173" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 45275 } } peers { permanent_uuid: "7dc50b1b1f0b4075bfca22384b64e2e3" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 33199 } } peers { permanent_uuid: "b5ad989a5dfa480db4819077922f1f75" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37585 } }
I20260811 06:30:02.878947 5421 ts_tablet_manager.cc:1428] T 75361ec40f3d46c493fbcf291acb97d4 P 7dc50b1b1f0b4075bfca22384b64e2e3: Time spent starting tablet: real 0.011s user 0.004s sys 0.006s
I20260811 06:30:02.881347 5423 raft_consensus.cc:348] T 75361ec40f3d46c493fbcf291acb97d4 P 3f35a41c31614632b92d2d071a976173 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f35a41c31614632b92d2d071a976173" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 45275 } } peers { permanent_uuid: "7dc50b1b1f0b4075bfca22384b64e2e3" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 33199 } } peers { permanent_uuid: "b5ad989a5dfa480db4819077922f1f75" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37585 } }
I20260811 06:30:02.881346 5422 raft_consensus.cc:348] T 75361ec40f3d46c493fbcf291acb97d4 P b5ad989a5dfa480db4819077922f1f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f35a41c31614632b92d2d071a976173" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 45275 } } peers { permanent_uuid: "7dc50b1b1f0b4075bfca22384b64e2e3" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 33199 } } peers { permanent_uuid: "b5ad989a5dfa480db4819077922f1f75" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37585 } }
I20260811 06:30:02.881851 5422 raft_consensus.cc:374] T 75361ec40f3d46c493fbcf291acb97d4 P b5ad989a5dfa480db4819077922f1f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:30:02.881851 5423 raft_consensus.cc:374] T 75361ec40f3d46c493fbcf291acb97d4 P 3f35a41c31614632b92d2d071a976173 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:30:02.881965 5423 raft_consensus.cc:729] T 75361ec40f3d46c493fbcf291acb97d4 P 3f35a41c31614632b92d2d071a976173 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f35a41c31614632b92d2d071a976173, State: Initialized, Role: FOLLOWER
I20260811 06:30:02.881978 5422 raft_consensus.cc:729] T 75361ec40f3d46c493fbcf291acb97d4 P b5ad989a5dfa480db4819077922f1f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5ad989a5dfa480db4819077922f1f75, State: Initialized, Role: FOLLOWER
I20260811 06:30:02.882618 5422 consensus_queue.cc:260] T 75361ec40f3d46c493fbcf291acb97d4 P b5ad989a5dfa480db4819077922f1f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f35a41c31614632b92d2d071a976173" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 45275 } } peers { permanent_uuid: "7dc50b1b1f0b4075bfca22384b64e2e3" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 33199 } } peers { permanent_uuid: "b5ad989a5dfa480db4819077922f1f75" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37585 } }
I20260811 06:30:02.882608 5423 consensus_queue.cc:260] T 75361ec40f3d46c493fbcf291acb97d4 P 3f35a41c31614632b92d2d071a976173 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f35a41c31614632b92d2d071a976173" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 45275 } } peers { permanent_uuid: "7dc50b1b1f0b4075bfca22384b64e2e3" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 33199 } } peers { permanent_uuid: "b5ad989a5dfa480db4819077922f1f75" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37585 } }
I20260811 06:30:02.884308 5392 heartbeater.cc:503] Master 127.28.239.254:33401 was elected leader, sending a full tablet report...
I20260811 06:30:02.884552 5422 ts_tablet_manager.cc:1428] T 75361ec40f3d46c493fbcf291acb97d4 P b5ad989a5dfa480db4819077922f1f75: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260811 06:30:02.884888 5423 ts_tablet_manager.cc:1428] T 75361ec40f3d46c493fbcf291acb97d4 P 3f35a41c31614632b92d2d071a976173: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260811 06:30:02.885402 5186 heartbeater.cc:503] Master 127.28.239.254:33401 was elected leader, sending a full tablet report...
W20260811 06:30:03.026396 5394 tablet.cc:2369] T 75361ec40f3d46c493fbcf291acb97d4 P 3f35a41c31614632b92d2d071a976173: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 06:30:03.042285 5188 tablet.cc:2369] T 75361ec40f3d46c493fbcf291acb97d4 P b5ad989a5dfa480db4819077922f1f75: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 06:30:03.064018 4992 tablet.cc:2369] T 75361ec40f3d46c493fbcf291acb97d4 P 7dc50b1b1f0b4075bfca22384b64e2e3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:30:03.080154 5434 raft_consensus.cc:482] T 75361ec40f3d46c493fbcf291acb97d4 P 3f35a41c31614632b92d2d071a976173 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:30:03.080612 5434 raft_consensus.cc:504] T 75361ec40f3d46c493fbcf291acb97d4 P 3f35a41c31614632b92d2d071a976173 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f35a41c31614632b92d2d071a976173" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 45275 } } peers { permanent_uuid: "7dc50b1b1f0b4075bfca22384b64e2e3" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 33199 } } peers { permanent_uuid: "b5ad989a5dfa480db4819077922f1f75" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37585 } }
I20260811 06:30:03.082667 5434 leader_election.cc:288] T 75361ec40f3d46c493fbcf291acb97d4 P 3f35a41c31614632b92d2d071a976173 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7dc50b1b1f0b4075bfca22384b64e2e3 (127.28.239.193:33199), b5ad989a5dfa480db4819077922f1f75 (127.28.239.194:37585)
I20260811 06:30:03.085837 5399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:03.082982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.194:37585 (local address 127.28.239.195:56851)
0811 06:30:03.083399 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:03.083418 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:03.083518 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:03.083998 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:03.084004 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:03.084034 (+ 30us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:03.084837 (+ 803us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 06:30:03.084841 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:03.084866 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 06:30:03.084930 (+ 64us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:30:03.085349 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:30:03.085356 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:30:03.085365 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 06:30:03.085509 (+ 144us) client_negotiation.cc:240] Negotiation successful
0811 06:30:03.085590 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272}
I20260811 06:30:03.085942 5417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:03.083241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.195:56851 (local address 127.28.239.194:37585)
0811 06:30:03.083586 (+ 345us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:03.083595 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:03.083618 (+ 23us) server_negotiation.cc:407] Connection header received
0811 06:30:03.083708 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:03.083718 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:03.083794 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:03.083914 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:03.085116 (+ 1202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:03.085121 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:03.085124 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:30:03.085162 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:03.085203 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:30:03.085222 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:03.085348 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:03.085517 (+ 169us) server_negotiation.cc:299] Negotiation successful
0811 06:30:03.085628 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194}
I20260811 06:30:03.086608 5420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:03.083163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.195:35993 (local address 127.28.239.193:33199)
0811 06:30:03.083396 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:03.083404 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:03.084308 (+ 904us) server_negotiation.cc:407] Connection header received
0811 06:30:03.084458 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:03.084464 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:03.084528 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:03.084632 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:03.085742 (+ 1110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:03.085748 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:03.085753 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:30:03.085773 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:03.085813 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:30:03.085828 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:03.085963 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:03.086277 (+ 314us) server_negotiation.cc:299] Negotiation successful
0811 06:30:03.086343 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 06:30:03.086778 5437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:03.083288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.193:33199 (local address 127.28.239.195:35993)
0811 06:30:03.084150 (+ 862us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:03.084172 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:03.084309 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:03.084722 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:03.084749 (+ 27us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:03.084785 (+ 36us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:03.085499 (+ 714us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 06:30:03.085502 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:03.085519 (+ 17us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 06:30:03.085589 (+ 70us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:30:03.086091 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:30:03.086099 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:30:03.086107 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 06:30:03.086257 (+ 150us) client_negotiation.cc:240] Negotiation successful
0811 06:30:03.086312 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":709,"thread_start_us":475,"threads_started":1}
I20260811 06:30:03.087188 5120 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75361ec40f3d46c493fbcf291acb97d4" candidate_uuid: "3f35a41c31614632b92d2d071a976173" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5ad989a5dfa480db4819077922f1f75" is_pre_election: true
I20260811 06:30:03.087891 5120 raft_consensus.cc:2454] T 75361ec40f3d46c493fbcf291acb97d4 P b5ad989a5dfa480db4819077922f1f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f35a41c31614632b92d2d071a976173 in term 0.
I20260811 06:30:03.088093 4926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75361ec40f3d46c493fbcf291acb97d4" candidate_uuid: "3f35a41c31614632b92d2d071a976173" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dc50b1b1f0b4075bfca22384b64e2e3" is_pre_election: true
I20260811 06:30:03.088725 4926 raft_consensus.cc:2454] T 75361ec40f3d46c493fbcf291acb97d4 P 7dc50b1b1f0b4075bfca22384b64e2e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f35a41c31614632b92d2d071a976173 in term 0.
I20260811 06:30:03.088690 5219 leader_election.cc:302] T 75361ec40f3d46c493fbcf291acb97d4 P 3f35a41c31614632b92d2d071a976173 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f35a41c31614632b92d2d071a976173, b5ad989a5dfa480db4819077922f1f75; no voters:
I20260811 06:30:03.089159 5434 raft_consensus.cc:2781] T 75361ec40f3d46c493fbcf291acb97d4 P 3f35a41c31614632b92d2d071a976173 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:30:03.089385 5434 raft_consensus.cc:482] T 75361ec40f3d46c493fbcf291acb97d4 P 3f35a41c31614632b92d2d071a976173 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:30:03.089521 5434 raft_consensus.cc:3037] T 75361ec40f3d46c493fbcf291acb97d4 P 3f35a41c31614632b92d2d071a976173 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:30:03.091813 5434 raft_consensus.cc:504] T 75361ec40f3d46c493fbcf291acb97d4 P 3f35a41c31614632b92d2d071a976173 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f35a41c31614632b92d2d071a976173" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 45275 } } peers { permanent_uuid: "7dc50b1b1f0b4075bfca22384b64e2e3" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 33199 } } peers { permanent_uuid: "b5ad989a5dfa480db4819077922f1f75" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37585 } }
I20260811 06:30:03.092412 5434 leader_election.cc:288] T 75361ec40f3d46c493fbcf291acb97d4 P 3f35a41c31614632b92d2d071a976173 [CANDIDATE]: Term 1 election: Requested vote from peers 7dc50b1b1f0b4075bfca22384b64e2e3 (127.28.239.193:33199), b5ad989a5dfa480db4819077922f1f75 (127.28.239.194:37585)
I20260811 06:30:03.092934 4926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75361ec40f3d46c493fbcf291acb97d4" candidate_uuid: "3f35a41c31614632b92d2d071a976173" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dc50b1b1f0b4075bfca22384b64e2e3"
I20260811 06:30:03.092962 5120 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75361ec40f3d46c493fbcf291acb97d4" candidate_uuid: "3f35a41c31614632b92d2d071a976173" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5ad989a5dfa480db4819077922f1f75"
I20260811 06:30:03.093147 4926 raft_consensus.cc:3037] T 75361ec40f3d46c493fbcf291acb97d4 P 7dc50b1b1f0b4075bfca22384b64e2e3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:30:03.093223 5120 raft_consensus.cc:3037] T 75361ec40f3d46c493fbcf291acb97d4 P b5ad989a5dfa480db4819077922f1f75 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:30:03.095533 4926 raft_consensus.cc:2454] T 75361ec40f3d46c493fbcf291acb97d4 P 7dc50b1b1f0b4075bfca22384b64e2e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f35a41c31614632b92d2d071a976173 in term 1.
I20260811 06:30:03.095541 5120 raft_consensus.cc:2454] T 75361ec40f3d46c493fbcf291acb97d4 P b5ad989a5dfa480db4819077922f1f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f35a41c31614632b92d2d071a976173 in term 1.
I20260811 06:30:03.095980 5216 leader_election.cc:302] T 75361ec40f3d46c493fbcf291acb97d4 P 3f35a41c31614632b92d2d071a976173 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f35a41c31614632b92d2d071a976173, 7dc50b1b1f0b4075bfca22384b64e2e3; no voters:
I20260811 06:30:03.096324 5434 raft_consensus.cc:2781] T 75361ec40f3d46c493fbcf291acb97d4 P 3f35a41c31614632b92d2d071a976173 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:30:03.096779 5434 raft_consensus.cc:686] T 75361ec40f3d46c493fbcf291acb97d4 P 3f35a41c31614632b92d2d071a976173 [term 1 LEADER]: Becoming Leader. State: Replica: 3f35a41c31614632b92d2d071a976173, State: Running, Role: LEADER
I20260811 06:30:03.097288 5434 consensus_queue.cc:237] T 75361ec40f3d46c493fbcf291acb97d4 P 3f35a41c31614632b92d2d071a976173 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f35a41c31614632b92d2d071a976173" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 45275 } } peers { permanent_uuid: "7dc50b1b1f0b4075bfca22384b64e2e3" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 33199 } } peers { permanent_uuid: "b5ad989a5dfa480db4819077922f1f75" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37585 } }
I20260811 06:30:03.102717 4711 catalog_manager.cc:5495] T 75361ec40f3d46c493fbcf291acb97d4 P 3f35a41c31614632b92d2d071a976173 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3f35a41c31614632b92d2d071a976173 (127.28.239.195). New cstate: current_term: 1 leader_uuid: "3f35a41c31614632b92d2d071a976173" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f35a41c31614632b92d2d071a976173" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 45275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7dc50b1b1f0b4075bfca22384b64e2e3" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 33199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5ad989a5dfa480db4819077922f1f75" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37585 } health_report { overall_health: UNKNOWN } } }
I20260811 06:30:03.134711 5415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:03.132610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37564 (local address 127.28.239.195:45275)
0811 06:30:03.132787 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:03.132792 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:03.132956 (+ 164us) server_negotiation.cc:407] Connection header received
0811 06:30:03.133137 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:03.133142 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:03.133243 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:03.133330 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:03.134224 (+ 894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:03.134243 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:03.134246 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:30:03.134262 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:03.134295 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:30:03.134307 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:03.134408 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:03.134475 (+ 67us) server_negotiation.cc:299] Negotiation successful
0811 06:30:03.134521 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 06:30:03.143383 4926 raft_consensus.cc:1261] T 75361ec40f3d46c493fbcf291acb97d4 P 7dc50b1b1f0b4075bfca22384b64e2e3 [term 1 FOLLOWER]: Refusing update from remote peer 3f35a41c31614632b92d2d071a976173: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:30:03.143381 5120 raft_consensus.cc:1261] T 75361ec40f3d46c493fbcf291acb97d4 P b5ad989a5dfa480db4819077922f1f75 [term 1 FOLLOWER]: Refusing update from remote peer 3f35a41c31614632b92d2d071a976173: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:30:03.144380 5439 consensus_queue.cc:1035] T 75361ec40f3d46c493fbcf291acb97d4 P 3f35a41c31614632b92d2d071a976173 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7dc50b1b1f0b4075bfca22384b64e2e3" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 33199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:30:03.144742 5434 consensus_queue.cc:1035] T 75361ec40f3d46c493fbcf291acb97d4 P 3f35a41c31614632b92d2d071a976173 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5ad989a5dfa480db4819077922f1f75" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:30:03.158079 5442 mvcc.cc:205] Tried to move back new op lower bound from 7317216473664610304 to 7317216473491005440. Current Snapshot: MvccSnapshot[applied={T|T < 7317216473664610304}]
I20260811 06:30:03.171687 4710 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:59342:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:30:03.172003 4710 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:59342:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:30:03.175658 4710 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1db3b579ecf949a38d2d108ae42a9b53: Sending DeleteTablet for 3 replicas of tablet 75361ec40f3d46c493fbcf291acb97d4
I20260811 06:30:03.176939 4895 tablet_service.cc:1514] Processing DeleteTablet for tablet 75361ec40f3d46c493fbcf291acb97d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:30:03 UTC) from {username='slave'} at 127.0.0.1:43208
I20260811 06:30:03.177074 5293 tablet_service.cc:1514] Processing DeleteTablet for tablet 75361ec40f3d46c493fbcf291acb97d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:30:03 UTC) from {username='slave'} at 127.0.0.1:37554
I20260811 06:30:03.177806 5088 tablet_service.cc:1514] Processing DeleteTablet for tablet 75361ec40f3d46c493fbcf291acb97d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:30:03 UTC) from {username='slave'} at 127.0.0.1:60372
I20260811 06:30:03.178651 5460 tablet_replica.cc:330] T 75361ec40f3d46c493fbcf291acb97d4 P 3f35a41c31614632b92d2d071a976173: stopping tablet replica
I20260811 06:30:03.179256 5460 raft_consensus.cc:2229] T 75361ec40f3d46c493fbcf291acb97d4 P 3f35a41c31614632b92d2d071a976173 [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:30:03.179885 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 4801
I20260811 06:30:03.180227 5460 raft_consensus.cc:2258] T 75361ec40f3d46c493fbcf291acb97d4 P 3f35a41c31614632b92d2d071a976173 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:30:03.181263 5458 tablet_replica.cc:330] T 75361ec40f3d46c493fbcf291acb97d4 P 7dc50b1b1f0b4075bfca22384b64e2e3: stopping tablet replica
I20260811 06:30:03.181774 5458 raft_consensus.cc:2229] T 75361ec40f3d46c493fbcf291acb97d4 P 7dc50b1b1f0b4075bfca22384b64e2e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:30:03.182284 5458 raft_consensus.cc:2258] T 75361ec40f3d46c493fbcf291acb97d4 P 7dc50b1b1f0b4075bfca22384b64e2e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:30:03.184055 5458 ts_tablet_manager.cc:1905] T 75361ec40f3d46c493fbcf291acb97d4 P 7dc50b1b1f0b4075bfca22384b64e2e3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:30:03.186430 5460 ts_tablet_manager.cc:1905] T 75361ec40f3d46c493fbcf291acb97d4 P 3f35a41c31614632b92d2d071a976173: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:30:03.189460 5420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:03.180989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43224 (local address 127.28.239.193:33199)
0811 06:30:03.181195 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:03.181202 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:03.186951 (+ 5749us) server_negotiation.cc:407] Connection header received
0811 06:30:03.187189 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:03.187195 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:03.187319 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:03.187435 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:03.188622 (+ 1187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:03.188634 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:03.188639 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:30:03.188659 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:03.188695 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:30:03.188710 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:03.188824 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:03.189072 (+ 248us) server_negotiation.cc:299] Negotiation successful
0811 06:30:03.189144 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 06:30:03.190790 4984 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:30:03.190814 5458 ts_tablet_manager.cc:1918] T 75361ec40f3d46c493fbcf291acb97d4 P 7dc50b1b1f0b4075bfca22384b64e2e3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:30:03.191069 5458 log.cc:1196] T 75361ec40f3d46c493fbcf291acb97d4 P 7dc50b1b1f0b4075bfca22384b64e2e3: Deleting WAL directory at /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429755927508-29631-0/minicluster-data/ts-0/wal/wals/75361ec40f3d46c493fbcf291acb97d4
I20260811 06:30:03.191771 5463 tablet_replica.cc:330] T 75361ec40f3d46c493fbcf291acb97d4 P b5ad989a5dfa480db4819077922f1f75: stopping tablet replica
I20260811 06:30:03.192391 5463 raft_consensus.cc:2229] T 75361ec40f3d46c493fbcf291acb97d4 P b5ad989a5dfa480db4819077922f1f75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:30:03.193022 5463 raft_consensus.cc:2258] T 75361ec40f3d46c493fbcf291acb97d4 P b5ad989a5dfa480db4819077922f1f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:30:03.195312 5463 ts_tablet_manager.cc:1905] T 75361ec40f3d46c493fbcf291acb97d4 P b5ad989a5dfa480db4819077922f1f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:30:03.201467 5463 ts_tablet_manager.cc:1918] T 75361ec40f3d46c493fbcf291acb97d4 P b5ad989a5dfa480db4819077922f1f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:30:03.201707 5463 log.cc:1196] T 75361ec40f3d46c493fbcf291acb97d4 P b5ad989a5dfa480db4819077922f1f75: Deleting WAL directory at /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429755927508-29631-0/minicluster-data/ts-1/wal/wals/75361ec40f3d46c493fbcf291acb97d4
I20260811 06:30:03.202346 5463 ts_tablet_manager.cc:1939] T 75361ec40f3d46c493fbcf291acb97d4 P b5ad989a5dfa480db4819077922f1f75: Deleting consensus metadata
I20260811 06:30:03.203510 5460 ts_tablet_manager.cc:1918] T 75361ec40f3d46c493fbcf291acb97d4 P 3f35a41c31614632b92d2d071a976173: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:30:03.203779 5460 log.cc:1196] T 75361ec40f3d46c493fbcf291acb97d4 P 3f35a41c31614632b92d2d071a976173: Deleting WAL directory at /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786429755927508-29631-0/minicluster-data/ts-2/wal/wals/75361ec40f3d46c493fbcf291acb97d4
I20260811 06:30:03.204541 5460 ts_tablet_manager.cc:1939] T 75361ec40f3d46c493fbcf291acb97d4 P 3f35a41c31614632b92d2d071a976173: Deleting consensus metadata
I20260811 06:30:03.204586 4688 catalog_manager.cc:4841] TS b5ad989a5dfa480db4819077922f1f75 (127.28.239.194:37585): tablet 75361ec40f3d46c493fbcf291acb97d4 (table test-table [id=9e276f2376244401bafdc63b26553d35]) successfully deleted
I20260811 06:30:03.206501 4687 catalog_manager.cc:4841] TS 3f35a41c31614632b92d2d071a976173 (127.28.239.195:45275): tablet 75361ec40f3d46c493fbcf291acb97d4 (table test-table [id=9e276f2376244401bafdc63b26553d35]) successfully deleted
I20260811 06:30:03.332841 5458 ts_tablet_manager.cc:1939] T 75361ec40f3d46c493fbcf291acb97d4 P 7dc50b1b1f0b4075bfca22384b64e2e3: Deleting consensus metadata
I20260811 06:30:03.332959 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 4801
W20260811 06:30:03.358942 4687 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.28.239.193:33199 (error 108)
I20260811 06:30:03.359133 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 4997
I20260811 06:30:03.362159 5472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:03.360839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.193:33199 (local address 127.0.0.1:43240)
0811 06:30:03.361794 (+ 955us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:03.361924 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.193:33199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":805,"thread_start_us":441,"threads_started":1}
W20260811 06:30:03.362421 4687 catalog_manager.cc:4584] TS 7dc50b1b1f0b4075bfca22384b64e2e3 (127.28.239.193:33199): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 75361ec40f3d46c493fbcf291acb97d4: Network error: Client connection negotiation failed: client connection to 127.28.239.193:33199: connect: Connection refused (error 111)
I20260811 06:30:03.363131 5417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:03.360334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60388 (local address 127.28.239.194:37585)
0811 06:30:03.360578 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:03.360587 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:03.360609 (+ 22us) server_negotiation.cc:407] Connection header received
0811 06:30:03.360688 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:03.360697 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:03.360826 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:03.360955 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:03.362148 (+ 1193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:03.362157 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:03.362162 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:30:03.362186 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:03.362231 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:30:03.362251 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:03.362418 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:03.362726 (+ 308us) server_negotiation.cc:299] Negotiation successful
0811 06:30:03.362797 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260811 06:30:03.363925 5178 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:30:03.416690 5472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:03.416065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.193:33199 (local address 127.0.0.1:43256)
0811 06:30:03.416333 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:03.416475 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.193:33199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260811 06:30:03.418058 5472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:03.417583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.193:33199 (local address 127.0.0.1:43262)
0811 06:30:03.417767 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:03.417876 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.193:33199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260811 06:30:03.457973 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 4997
I20260811 06:30:03.478870 5472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:03.478237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.193:33199 (local address 127.0.0.1:43268)
0811 06:30:03.478529 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:03.478658 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.193:33199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260811 06:30:03.480392 5472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:03.479787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.193:33199 (local address 127.0.0.1:43276)
0811 06:30:03.480046 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:03.480163 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.193:33199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260811 06:30:03.485738 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 5190
I20260811 06:30:03.488919 5415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:03.486654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37570 (local address 127.28.239.195:45275)
0811 06:30:03.486896 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:03.486902 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:03.486923 (+ 21us) server_negotiation.cc:407] Connection header received
0811 06:30:03.487106 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:03.487111 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:03.487173 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:03.487287 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:03.488229 (+ 942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:03.488236 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:03.488240 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:30:03.488266 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:03.488298 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:30:03.488310 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:03.488466 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:03.488529 (+ 63us) server_negotiation.cc:299] Negotiation successful
0811 06:30:03.488598 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
I20260811 06:30:03.489732 5385 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:30:03.561466 5472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:03.560871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.193:33199 (local address 127.0.0.1:43286)
0811 06:30:03.561183 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:03.561299 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.193:33199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116}
I20260811 06:30:03.562744 5472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:03.562291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.193:33199 (local address 127.0.0.1:43288)
0811 06:30:03.562508 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:03.562585 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.193:33199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124}
I20260811 06:30:03.613806 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 5190
I20260811 06:30:03.645076 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 4661
I20260811 06:30:03.646425 4759 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:30:03.755108 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 4661
2026-08-11T06:30:03Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (3039 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260811 06:30:03.805505 29631 mini_oidc.cc:152] Starting JWKS server
I20260811 06:30:03.808575 29631 webserver.cc:460] Webserver started at https://127.0.0.1:33893/ using document root <none> and password file <none>
I20260811 06:30:03.808857 29631 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 06:30:03.809474 29631 webserver.cc:460] Webserver started at http://127.0.0.1:33585/ using document root <none> and password file <none>
2026-08-11T06:30:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:30:03Z Disabled control of system clock
I20260811 06:30:03.844393 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786429755927508-29631-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786429755927508-29631-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786429755927508-29631-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:40729
--webserver_interface=127.28.239.254
--webserver_port=0
--builtin_ntp_servers=127.28.239.212:38585
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.239.254:40729
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33893/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786429755927508-29631-0/testchainca.cert with env {}
W20260811 06:30:04.185990 5485 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:30:04.186393 5485 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:30:04.186566 5485 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:30:04.194190 5485 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:30:04.194361 5485 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:30:04.194458 5485 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:30:04.194571 5485 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:30:04.194666 5485 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 06:30:04.194790 5485 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 06:30:04.194857 5485 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33893/jwks.json
I20260811 06:30:04.205996 5485 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:38585
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786429755927508-29631-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786429755927508-29631-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.239.254:40729
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:40729
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33893/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.239.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786429755927508-29631-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786429755927508-29631-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:30:04.207494 5485 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:30:04.208915 5485 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:30:04.221884 5490 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:30:04.222018 5501 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:30:04.222048 5494 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:30:04.223207 5485 server_base.cc:1034] running on GCE node
I20260811 06:30:04.224236 5485 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:30:04.226141 5485 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:30:04.227420 5485 hybrid_clock.cc:648] HybridClock initialized: now 1786429804227383 us; error 64 us; skew 500 ppm
I20260811 06:30:04.227869 5485 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:30:04.230417 5485 webserver.cc:460] Webserver started at http://127.28.239.254:44151/ using document root <none> and password file <none>
I20260811 06:30:04.231159 5485 fs_manager.cc:359] Metadata directory not provided
I20260811 06:30:04.231303 5485 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:30:04.231659 5485 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:30:04.234844 5485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786429755927508-29631-0/minicluster-data/master-0/data/instance:
uuid: "0c7e652576f14a96b1438281fef08e5e"
format_stamp: "Formatted at 2026-08-11 06:30:04 on dist-test-slave-gkw7"
server_key: "7c54c424f284f11155196d2efce2dad6"
server_key_iv: "66925202295aba7a795fc0b13385b598"
server_key_version: "encryptionkey@0"
I20260811 06:30:04.235716 5485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786429755927508-29631-0/minicluster-data/master-0/wal/instance:
uuid: "0c7e652576f14a96b1438281fef08e5e"
format_stamp: "Formatted at 2026-08-11 06:30:04 on dist-test-slave-gkw7"
server_key: "7c54c424f284f11155196d2efce2dad6"
server_key_iv: "66925202295aba7a795fc0b13385b598"
server_key_version: "encryptionkey@0"
I20260811 06:30:04.241518 5485 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 06:30:04.245441 5510 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:04.247088 5485 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 06:30:04.247316 5485 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786429755927508-29631-0/minicluster-data/master-0/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786429755927508-29631-0/minicluster-data/master-0/wal
uuid: "0c7e652576f14a96b1438281fef08e5e"
format_stamp: "Formatted at 2026-08-11 06:30:04 on dist-test-slave-gkw7"
server_key: "7c54c424f284f11155196d2efce2dad6"
server_key_iv: "66925202295aba7a795fc0b13385b598"
server_key_version: "encryptionkey@0"
I20260811 06:30:04.247565 5485 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786429755927508-29631-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786429755927508-29631-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786429755927508-29631-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:30:04.266448 5485 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:30:04.267380 5485 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:30:04.267730 5485 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:30:04.287469 5485 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.254:40729
I20260811 06:30:04.287477 5593 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.254:40729 every 8 connection(s)
I20260811 06:30:04.288931 5485 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
I20260811 06:30:04.292929 5594 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:30:04.297032 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 5485
I20260811 06:30:04.297284 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786429755927508-29631-0/minicluster-data/master-0/wal/instance
I20260811 06:30:04.298187 29631 external_mini_cluster.cc:1428] Setting key 567eee0ed8aedb3b7f334704d6c8f0fc
I20260811 06:30:04.306679 5594 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c7e652576f14a96b1438281fef08e5e: Bootstrap starting.
I20260811 06:30:04.310204 5594 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0c7e652576f14a96b1438281fef08e5e: Neither blocks nor log segments found. Creating new log.
I20260811 06:30:04.311411 5594 log.cc:824] T 00000000000000000000000000000000 P 0c7e652576f14a96b1438281fef08e5e: Log is configured to *not* fsync() on all Append() calls
I20260811 06:30:04.312251 5598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:04.300343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40706 (local address 127.28.239.254:40729)
0811 06:30:04.301409 (+ 1066us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:04.301421 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:04.301462 (+ 41us) server_negotiation.cc:407] Connection header received
0811 06:30:04.303145 (+ 1683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:04.303168 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:04.303673 (+ 505us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:04.304090 (+ 417us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:04.304735 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:04.306068 (+ 1333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:04.307518 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:04.308202 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:04.308439 (+ 237us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:04.309704 (+ 1265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:04.309712 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:04.309729 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:30:04.310305 (+ 576us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:04.310456 (+ 151us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:30:04.310486 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:04.310630 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:04.310883 (+ 253us) server_negotiation.cc:299] Negotiation successful
0811 06:30:04.311204 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":624,"thread_start_us":385,"threads_started":1}
I20260811 06:30:04.314419 5594 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c7e652576f14a96b1438281fef08e5e: No bootstrap required, opened a new log
I20260811 06:30:04.316083 29631 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 06:30:04.322288 5594 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0c7e652576f14a96b1438281fef08e5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c7e652576f14a96b1438281fef08e5e" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 40729 } }
I20260811 06:30:04.322680 5594 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0c7e652576f14a96b1438281fef08e5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:30:04.322795 5594 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0c7e652576f14a96b1438281fef08e5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c7e652576f14a96b1438281fef08e5e, State: Initialized, Role: FOLLOWER
I20260811 06:30:04.323375 5594 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0c7e652576f14a96b1438281fef08e5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c7e652576f14a96b1438281fef08e5e" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 40729 } }
I20260811 06:30:04.323623 5594 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0c7e652576f14a96b1438281fef08e5e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:30:04.323796 5594 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0c7e652576f14a96b1438281fef08e5e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:30:04.323977 5594 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0c7e652576f14a96b1438281fef08e5e [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:30:04.326511 5594 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0c7e652576f14a96b1438281fef08e5e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c7e652576f14a96b1438281fef08e5e" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 40729 } }
I20260811 06:30:04.327045 5594 leader_election.cc:302] T 00000000000000000000000000000000 P 0c7e652576f14a96b1438281fef08e5e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0c7e652576f14a96b1438281fef08e5e; no voters:
I20260811 06:30:04.327663 5594 leader_election.cc:288] T 00000000000000000000000000000000 P 0c7e652576f14a96b1438281fef08e5e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:30:04.327818 5602 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0c7e652576f14a96b1438281fef08e5e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:30:04.328564 5602 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0c7e652576f14a96b1438281fef08e5e [term 1 LEADER]: Becoming Leader. State: Replica: 0c7e652576f14a96b1438281fef08e5e, State: Running, Role: LEADER
I20260811 06:30:04.329171 5602 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0c7e652576f14a96b1438281fef08e5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c7e652576f14a96b1438281fef08e5e" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 40729 } }
I20260811 06:30:04.329949 5594 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0c7e652576f14a96b1438281fef08e5e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:30:04.332336 5604 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c7e652576f14a96b1438281fef08e5e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0c7e652576f14a96b1438281fef08e5e. Latest consensus state: current_term: 1 leader_uuid: "0c7e652576f14a96b1438281fef08e5e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c7e652576f14a96b1438281fef08e5e" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 40729 } } }
I20260811 06:30:04.332784 5604 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c7e652576f14a96b1438281fef08e5e [sys.catalog]: This master's current role is: LEADER
I20260811 06:30:04.333307 5603 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c7e652576f14a96b1438281fef08e5e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0c7e652576f14a96b1438281fef08e5e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c7e652576f14a96b1438281fef08e5e" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 40729 } } }
I20260811 06:30:04.333653 5603 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c7e652576f14a96b1438281fef08e5e [sys.catalog]: This master's current role is: LEADER
I20260811 06:30:04.338337 5611 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:30:04.343600 5611 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:30:04.353099 5611 catalog_manager.cc:1347] Generated new cluster ID: 95bb2889a51349359b543d0d9a4961b4
I20260811 06:30:04.353258 5611 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:30:04.375147 5611 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:30:04.377159 5611 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:30:04.393183 5611 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0c7e652576f14a96b1438281fef08e5e: Generated new TSK 0
I20260811 06:30:04.394382 5611 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 06:30:04.435882 5637 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260811 06:30:04.436214 5637 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 06:30:04.438650 5598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:04.413791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40720 (local address 127.28.239.254:40729)
0811 06:30:04.414008 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:04.414017 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:04.414509 (+ 492us) server_negotiation.cc:407] Connection header received
0811 06:30:04.414950 (+ 441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:04.414954 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:04.415075 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:04.415273 (+ 198us) server_negotiation.cc:226] Negotiated authn=JWT
0811 06:30:04.416115 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:04.416888 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:04.418463 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:04.418984 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:04.419161 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:04.419767 (+ 606us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 06:30:04.437887 (+ 18120us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33893/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 06:30:04.438317 (+ 430us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:40720: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33893/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":59}
W20260811 06:30:04.438800 5598 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:40720: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33893/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260811 06:30:04.440464 5636 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:30:04.413720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:40729 (local address 127.0.0.1:40720)
0811 06:30:04.414395 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:04.414411 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:04.414621 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:04.415493 (+ 872us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:04.415501 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:04.415516 (+ 15us) client_negotiation.cc:189] Negotiated authn=JWT
0811 06:30:04.415931 (+ 415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:04.415943 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:04.417052 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:04.417058 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:04.418308 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:04.418320 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:04.419180 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:04.419187 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:04.419390 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:04.419454 (+ 64us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 06:30:04.438249 (+ 18795us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33893/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 06:30:04.438633 (+ 384us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.28.239.254:40729: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33893/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":375,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.28.239.254:40729: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33893/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260811 06:30:04.442781 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 5485
I20260811 06:30:04.443956 5586 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:30:04.540940 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 5485
2026-08-11T06:30:04Z chronyd exiting
I20260811 06:30:04.603693 29631 test_util.cc:175] -----------------------------------------------
I20260811 06:30:04.603895 29631 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786429755927508-29631-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (803 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260811 06:30:04.608801 29631 mini_oidc.cc:152] Starting JWKS server
I20260811 06:30:04.610873 29631 webserver.cc:460] Webserver started at https://127.0.0.1:39131/ using document root <none> and password file <none>
I20260811 06:30:04.611126 29631 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 06:30:04.611832 29631 webserver.cc:460] Webserver started at http://127.0.0.1:32999/ using document root <none> and password file <none>
2026-08-11T06:30:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:30:04Z Disabled control of system clock
I20260811 06:30:04.648279 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786429755927508-29631-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786429755927508-29631-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786429755927508-29631-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:39921
--webserver_interface=127.28.239.254
--webserver_port=0
--builtin_ntp_servers=127.28.239.212:43513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.239.254:39921
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39131/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786429755927508-29631-0/testchainca.cert with env {}
W20260811 06:30:05.000133 5647 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:30:05.000499 5647 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:30:05.000689 5647 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:30:05.008255 5647 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:30:05.008383 5647 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:30:05.008504 5647 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:30:05.008617 5647 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:30:05.008692 5647 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 06:30:05.008797 5647 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 06:30:05.008863 5647 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:39131/jwks.json
I20260811 06:30:05.019519 5647 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:43513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786429755927508-29631-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786429755927508-29631-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.239.254:39921
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:39921
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39131/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.239.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786429755927508-29631-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786429755927508-29631-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:30:05.020938 5647 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:30:05.022346 5647 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:30:05.038074 5655 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:30:05.038115 5654 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:30:05.038144 5658 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:30:05.039018 5647 server_base.cc:1034] running on GCE node
I20260811 06:30:05.039906 5647 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:30:05.041599 5647 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:30:05.042871 5647 hybrid_clock.cc:648] HybridClock initialized: now 1786429805042802 us; error 50 us; skew 500 ppm
I20260811 06:30:05.043298 5647 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:30:05.050626 5647 webserver.cc:460] Webserver started at http://127.28.239.254:46185/ using document root <none> and password file <none>
I20260811 06:30:05.051445 5647 fs_manager.cc:359] Metadata directory not provided
I20260811 06:30:05.051599 5647 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:30:05.051954 5647 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:30:05.055094 5647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786429755927508-29631-0/minicluster-data/master-0/data/instance:
uuid: "4f56a1d2e5e3446a9e624ad3e321a251"
format_stamp: "Formatted at 2026-08-11 06:30:05 on dist-test-slave-gkw7"
server_key: "d3dc607858aa945b0778e8af645ee4a8"
server_key_iv: "879908e98bd750111865bb90961cd9c4"
server_key_version: "encryptionkey@0"
I20260811 06:30:05.055922 5647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786429755927508-29631-0/minicluster-data/master-0/wal/instance:
uuid: "4f56a1d2e5e3446a9e624ad3e321a251"
format_stamp: "Formatted at 2026-08-11 06:30:05 on dist-test-slave-gkw7"
server_key: "d3dc607858aa945b0778e8af645ee4a8"
server_key_iv: "879908e98bd750111865bb90961cd9c4"
server_key_version: "encryptionkey@0"
I20260811 06:30:05.061688 5647 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260811 06:30:05.065392 5668 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:05.066910 5647 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:30:05.067137 5647 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786429755927508-29631-0/minicluster-data/master-0/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786429755927508-29631-0/minicluster-data/master-0/wal
uuid: "4f56a1d2e5e3446a9e624ad3e321a251"
format_stamp: "Formatted at 2026-08-11 06:30:05 on dist-test-slave-gkw7"
server_key: "d3dc607858aa945b0778e8af645ee4a8"
server_key_iv: "879908e98bd750111865bb90961cd9c4"
server_key_version: "encryptionkey@0"
I20260811 06:30:05.067412 5647 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786429755927508-29631-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786429755927508-29631-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786429755927508-29631-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:30:05.083412 5647 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:30:05.084347 5647 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:30:05.084692 5647 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:30:05.104054 5647 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.254:39921
I20260811 06:30:05.104061 5749 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.254:39921 every 8 connection(s)
I20260811 06:30:05.105582 5647 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
I20260811 06:30:05.109449 5750 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:30:05.111083 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 5647
I20260811 06:30:05.111301 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786429755927508-29631-0/minicluster-data/master-0/wal/instance
I20260811 06:30:05.111995 29631 external_mini_cluster.cc:1428] Setting key f9f64a527280be712d52c2854e74ce82
I20260811 06:30:05.123697 5750 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f56a1d2e5e3446a9e624ad3e321a251: Bootstrap starting.
I20260811 06:30:05.128155 5750 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4f56a1d2e5e3446a9e624ad3e321a251: Neither blocks nor log segments found. Creating new log.
I20260811 06:30:05.128468 5753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:05.114074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41710 (local address 127.28.239.254:39921)
0811 06:30:05.115230 (+ 1156us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:05.115246 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:05.115301 (+ 55us) server_negotiation.cc:407] Connection header received
0811 06:30:05.117668 (+ 2367us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:05.117699 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:05.118322 (+ 623us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:05.118806 (+ 484us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:05.119397 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:05.121579 (+ 2182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:05.122929 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:05.124007 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:05.124200 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:05.125373 (+ 1173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:05.125385 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:05.125408 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:30:05.126260 (+ 852us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:05.126396 (+ 136us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:30:05.126425 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:05.126546 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:05.126763 (+ 217us) server_negotiation.cc:299] Negotiation successful
0811 06:30:05.127104 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":601,"thread_start_us":355,"threads_started":1}
I20260811 06:30:05.129489 5750 log.cc:824] T 00000000000000000000000000000000 P 4f56a1d2e5e3446a9e624ad3e321a251: Log is configured to *not* fsync() on all Append() calls
I20260811 06:30:05.132709 29631 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 06:30:05.132925 5750 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f56a1d2e5e3446a9e624ad3e321a251: No bootstrap required, opened a new log
I20260811 06:30:05.141206 5750 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4f56a1d2e5e3446a9e624ad3e321a251 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f56a1d2e5e3446a9e624ad3e321a251" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 39921 } }
I20260811 06:30:05.141681 5750 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4f56a1d2e5e3446a9e624ad3e321a251 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:30:05.141832 5750 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4f56a1d2e5e3446a9e624ad3e321a251 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f56a1d2e5e3446a9e624ad3e321a251, State: Initialized, Role: FOLLOWER
I20260811 06:30:05.142501 5750 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4f56a1d2e5e3446a9e624ad3e321a251 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f56a1d2e5e3446a9e624ad3e321a251" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 39921 } }
I20260811 06:30:05.142750 5750 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4f56a1d2e5e3446a9e624ad3e321a251 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:30:05.142907 5750 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4f56a1d2e5e3446a9e624ad3e321a251 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:30:05.143090 5750 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4f56a1d2e5e3446a9e624ad3e321a251 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:30:05.145291 5750 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4f56a1d2e5e3446a9e624ad3e321a251 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f56a1d2e5e3446a9e624ad3e321a251" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 39921 } }
I20260811 06:30:05.145888 5750 leader_election.cc:302] T 00000000000000000000000000000000 P 4f56a1d2e5e3446a9e624ad3e321a251 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4f56a1d2e5e3446a9e624ad3e321a251; no voters:
I20260811 06:30:05.146472 5750 leader_election.cc:288] T 00000000000000000000000000000000 P 4f56a1d2e5e3446a9e624ad3e321a251 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:30:05.146654 5755 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4f56a1d2e5e3446a9e624ad3e321a251 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:30:05.147915 5755 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4f56a1d2e5e3446a9e624ad3e321a251 [term 1 LEADER]: Becoming Leader. State: Replica: 4f56a1d2e5e3446a9e624ad3e321a251, State: Running, Role: LEADER
I20260811 06:30:05.148552 5755 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4f56a1d2e5e3446a9e624ad3e321a251 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f56a1d2e5e3446a9e624ad3e321a251" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 39921 } }
I20260811 06:30:05.148830 5750 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4f56a1d2e5e3446a9e624ad3e321a251 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:30:05.152067 5756 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f56a1d2e5e3446a9e624ad3e321a251 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4f56a1d2e5e3446a9e624ad3e321a251" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f56a1d2e5e3446a9e624ad3e321a251" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 39921 } } }
I20260811 06:30:05.152655 5756 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f56a1d2e5e3446a9e624ad3e321a251 [sys.catalog]: This master's current role is: LEADER
I20260811 06:30:05.152156 5757 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f56a1d2e5e3446a9e624ad3e321a251 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4f56a1d2e5e3446a9e624ad3e321a251. Latest consensus state: current_term: 1 leader_uuid: "4f56a1d2e5e3446a9e624ad3e321a251" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f56a1d2e5e3446a9e624ad3e321a251" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 39921 } } }
I20260811 06:30:05.152812 5757 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f56a1d2e5e3446a9e624ad3e321a251 [sys.catalog]: This master's current role is: LEADER
I20260811 06:30:05.158063 5766 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:30:05.163913 5766 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:30:05.172811 5766 catalog_manager.cc:1347] Generated new cluster ID: ae0723d379604e31b0087252acd639d4
I20260811 06:30:05.172956 5766 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:30:05.191783 5766 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:30:05.193135 5766 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:30:05.201035 5766 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4f56a1d2e5e3446a9e624ad3e321a251: Generated new TSK 0
I20260811 06:30:05.202159 5766 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 06:30:05.245486 5792 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260811 06:30:05.245651 5792 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 06:30:05.246950 5753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:05.226725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41726 (local address 127.28.239.254:39921)
0811 06:30:05.226941 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:05.226949 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:05.227420 (+ 471us) server_negotiation.cc:407] Connection header received
0811 06:30:05.227638 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:05.227644 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:05.227750 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:05.227936 (+ 186us) server_negotiation.cc:226] Negotiated authn=JWT
0811 06:30:05.228583 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:05.229393 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:05.230884 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:05.231344 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:05.231486 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:05.232045 (+ 559us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 06:30:05.246173 (+ 14128us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39131/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 06:30:05.246597 (+ 424us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41726: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39131/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":68}
W20260811 06:30:05.247098 5753 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41726: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39131/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260811 06:30:05.247129 5791 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:30:05.226543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:39921 (local address 127.0.0.1:41726)
0811 06:30:05.227301 (+ 758us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:05.227322 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:05.227454 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:05.228007 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:05.228015 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:05.228030 (+ 15us) client_negotiation.cc:189] Negotiated authn=JWT
0811 06:30:05.228392 (+ 362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:05.228411 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:05.229584 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:05.229590 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:05.230708 (+ 1118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:05.230721 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:05.231545 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:05.231553 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:05.231724 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:05.231759 (+ 35us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 06:30:05.246424 (+ 14665us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39131/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 06:30:05.246707 (+ 283us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.28.239.254:39921: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39131/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":597,"thread_start_us":446,"threads_started":1}
I20260811 06:30:05.248806 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 5647
I20260811 06:30:05.250348 5743 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:30:05.350390 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 5647
2026-08-11T06:30:05Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (809 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260811 06:30:05.418090 29631 mini_oidc.cc:152] Starting JWKS server
I20260811 06:30:05.420121 29631 webserver.cc:460] Webserver started at https://127.0.0.1:40161/ using document root <none> and password file <none>
I20260811 06:30:05.420428 29631 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 06:30:05.421080 29631 webserver.cc:460] Webserver started at http://127.0.0.1:40417/ using document root <none> and password file <none>
2026-08-11T06:30:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:30:05Z Disabled control of system clock
I20260811 06:30:05.458901 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786429755927508-29631-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786429755927508-29631-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786429755927508-29631-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:35947
--webserver_interface=127.28.239.254
--webserver_port=0
--builtin_ntp_servers=127.28.239.212:41375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.239.254:35947
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40161/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 06:30:05.801610 5799 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:30:05.801987 5799 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:30:05.802175 5799 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:30:05.809448 5799 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:30:05.809652 5799 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:30:05.809710 5799 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:30:05.809764 5799 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:30:05.809810 5799 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 06:30:05.809867 5799 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 06:30:05.809906 5799 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:40161/jwks.json
I20260811 06:30:05.819169 5799 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:41375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786429755927508-29631-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786429755927508-29631-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.239.254:35947
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:35947
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40161/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.239.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786429755927508-29631-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:30:05.820473 5799 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:30:05.822527 5799 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:30:05.834733 5806 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:30:05.834864 5807 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:30:05.834864 5810 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:30:05.835695 5799 server_base.cc:1034] running on GCE node
I20260811 06:30:05.836459 5799 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:30:05.837970 5799 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:30:05.839198 5799 hybrid_clock.cc:648] HybridClock initialized: now 1786429805839114 us; error 67 us; skew 500 ppm
I20260811 06:30:05.839543 5799 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:30:05.841987 5799 webserver.cc:460] Webserver started at http://127.28.239.254:37495/ using document root <none> and password file <none>
I20260811 06:30:05.842685 5799 fs_manager.cc:359] Metadata directory not provided
I20260811 06:30:05.842773 5799 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:30:05.843057 5799 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:30:05.846083 5799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786429755927508-29631-0/minicluster-data/master-0/data/instance:
uuid: "7b569269187d4167a187a76273c2732e"
format_stamp: "Formatted at 2026-08-11 06:30:05 on dist-test-slave-gkw7"
server_key: "9633120b89c38f04c0beb04f493aded7"
server_key_iv: "567f513a1be8f04866b9390366440768"
server_key_version: "encryptionkey@0"
I20260811 06:30:05.846855 5799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786429755927508-29631-0/minicluster-data/master-0/wal/instance:
uuid: "7b569269187d4167a187a76273c2732e"
format_stamp: "Formatted at 2026-08-11 06:30:05 on dist-test-slave-gkw7"
server_key: "9633120b89c38f04c0beb04f493aded7"
server_key_iv: "567f513a1be8f04866b9390366440768"
server_key_version: "encryptionkey@0"
I20260811 06:30:05.852162 5799 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 06:30:05.855695 5819 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:05.857117 5799 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 06:30:05.857316 5799 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786429755927508-29631-0/minicluster-data/master-0/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786429755927508-29631-0/minicluster-data/master-0/wal
uuid: "7b569269187d4167a187a76273c2732e"
format_stamp: "Formatted at 2026-08-11 06:30:05 on dist-test-slave-gkw7"
server_key: "9633120b89c38f04c0beb04f493aded7"
server_key_iv: "567f513a1be8f04866b9390366440768"
server_key_version: "encryptionkey@0"
I20260811 06:30:05.857569 5799 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786429755927508-29631-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786429755927508-29631-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786429755927508-29631-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:30:05.883320 5799 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:30:05.884282 5799 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:30:05.884636 5799 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:30:05.904336 5799 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.254:35947
I20260811 06:30:05.904381 5913 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.254:35947 every 8 connection(s)
I20260811 06:30:05.905875 5799 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
I20260811 06:30:05.909922 5914 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:30:05.914186 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 5799
I20260811 06:30:05.914435 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786429755927508-29631-0/minicluster-data/master-0/wal/instance
I20260811 06:30:05.915091 29631 external_mini_cluster.cc:1428] Setting key bc193821a3e9a52eea949a656310f4fd
I20260811 06:30:05.921440 5914 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b569269187d4167a187a76273c2732e: Bootstrap starting.
I20260811 06:30:05.925329 5914 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7b569269187d4167a187a76273c2732e: Neither blocks nor log segments found. Creating new log.
I20260811 06:30:05.926725 5914 log.cc:824] T 00000000000000000000000000000000 P 7b569269187d4167a187a76273c2732e: Log is configured to *not* fsync() on all Append() calls
I20260811 06:30:05.929978 5917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:05.917221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44976 (local address 127.28.239.254:35947)
0811 06:30:05.918324 (+ 1103us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:05.918335 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:05.918379 (+ 44us) server_negotiation.cc:407] Connection header received
0811 06:30:05.920121 (+ 1742us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:05.920145 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:05.920609 (+ 464us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:05.920940 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:05.921603 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:05.922965 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:05.924871 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:05.925724 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:05.925871 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:05.927135 (+ 1264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:05.927143 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:05.927157 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:30:05.928009 (+ 852us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:05.928143 (+ 134us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:30:05.928176 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:05.928289 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:05.928570 (+ 281us) server_negotiation.cc:299] Negotiation successful
0811 06:30:05.928925 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":648,"thread_start_us":421,"threads_started":1}
I20260811 06:30:05.930310 5914 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b569269187d4167a187a76273c2732e: No bootstrap required, opened a new log
I20260811 06:30:05.934119 29631 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 06:30:05.938634 5914 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7b569269187d4167a187a76273c2732e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b569269187d4167a187a76273c2732e" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 35947 } }
I20260811 06:30:05.939101 5914 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7b569269187d4167a187a76273c2732e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:30:05.939214 5914 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7b569269187d4167a187a76273c2732e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b569269187d4167a187a76273c2732e, State: Initialized, Role: FOLLOWER
I20260811 06:30:05.939823 5914 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7b569269187d4167a187a76273c2732e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b569269187d4167a187a76273c2732e" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 35947 } }
I20260811 06:30:05.940115 5914 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7b569269187d4167a187a76273c2732e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:30:05.940287 5914 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b569269187d4167a187a76273c2732e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:30:05.940506 5914 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7b569269187d4167a187a76273c2732e [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:30:05.943609 5914 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b569269187d4167a187a76273c2732e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b569269187d4167a187a76273c2732e" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 35947 } }
I20260811 06:30:05.944308 5914 leader_election.cc:302] T 00000000000000000000000000000000 P 7b569269187d4167a187a76273c2732e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7b569269187d4167a187a76273c2732e; no voters:
I20260811 06:30:05.944988 5914 leader_election.cc:288] T 00000000000000000000000000000000 P 7b569269187d4167a187a76273c2732e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:30:05.945147 5919 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7b569269187d4167a187a76273c2732e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:30:05.945994 5919 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7b569269187d4167a187a76273c2732e [term 1 LEADER]: Becoming Leader. State: Replica: 7b569269187d4167a187a76273c2732e, State: Running, Role: LEADER
I20260811 06:30:05.946552 5919 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7b569269187d4167a187a76273c2732e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b569269187d4167a187a76273c2732e" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 35947 } }
I20260811 06:30:05.947383 5914 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7b569269187d4167a187a76273c2732e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:30:05.949342 5920 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b569269187d4167a187a76273c2732e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7b569269187d4167a187a76273c2732e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b569269187d4167a187a76273c2732e" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 35947 } } }
I20260811 06:30:05.949775 5920 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b569269187d4167a187a76273c2732e [sys.catalog]: This master's current role is: LEADER
I20260811 06:30:05.949991 5921 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b569269187d4167a187a76273c2732e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7b569269187d4167a187a76273c2732e. Latest consensus state: current_term: 1 leader_uuid: "7b569269187d4167a187a76273c2732e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b569269187d4167a187a76273c2732e" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 35947 } } }
I20260811 06:30:05.950377 5921 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b569269187d4167a187a76273c2732e [sys.catalog]: This master's current role is: LEADER
I20260811 06:30:05.957718 5925 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:30:05.963624 5925 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:30:05.972545 5925 catalog_manager.cc:1347] Generated new cluster ID: 58d6b064f1d54d9d85ce770c0f537728
I20260811 06:30:05.972694 5925 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:30:05.981402 5925 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:30:05.982770 5925 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:30:05.989974 5925 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7b569269187d4167a187a76273c2732e: Generated new TSK 0
I20260811 06:30:05.990855 5925 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:30:06.020510 5917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:06.011756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44990 (local address 127.28.239.254:35947)
0811 06:30:06.011981 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:06.011988 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:06.012406 (+ 418us) server_negotiation.cc:407] Connection header received
0811 06:30:06.012638 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:06.012644 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:06.012736 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:06.012913 (+ 177us) server_negotiation.cc:226] Negotiated authn=JWT
0811 06:30:06.013387 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:06.014559 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:06.015918 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:06.016642 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:06.016774 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:06.017313 (+ 539us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 06:30:06.019820 (+ 2507us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40161/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 06:30:06.020144 (+ 324us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:44990: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40161/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":72}
W20260811 06:30:06.020663 5917 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:44990: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40161/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260811 06:30:06.022343 5957 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:30:06.011583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:35947 (local address 127.0.0.1:44990)
0811 06:30:06.012301 (+ 718us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:06.012316 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:06.012438 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:06.012936 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:06.012940 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:06.012949 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0811 06:30:06.013221 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:06.013248 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:06.014704 (+ 1456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:06.014708 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:06.015771 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:06.015782 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:06.016850 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:06.016855 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:06.016980 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:06.017014 (+ 34us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 06:30:06.020034 (+ 3020us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40161/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 06:30:06.021855 (+ 1821us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.28.239.254:35947: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40161/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":568,"mutex_wait_us":1493,"thread_start_us":417,"threads_started":1}
I20260811 06:30:06.023501 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 5799
I20260811 06:30:06.024849 5900 generic_service.cc:196] Checking for leaks (request via RPC)
E20260811 06:30:06.032999 5958 webserver.cc:533] Webserver: sslize failed:
E20260811 06:30:06.033095 5958 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 06:30:06.119146 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 5799
2026-08-11T06:30:06Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (767 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-11T06:30:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:30:06Z Disabled control of system clock
I20260811 06:30:06.220851 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786429755927508-29631-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786429755927508-29631-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786429755927508-29631-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:45073
--webserver_interface=127.28.239.254
--webserver_port=0
--builtin_ntp_servers=127.28.239.212:36579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.239.254:45073
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786429755927508-29631-0/testchainca.cert with env {}
W20260811 06:30:06.582391 5965 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:30:06.582759 5965 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:30:06.582958 5965 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:30:06.590827 5965 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:30:06.590942 5965 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:30:06.591071 5965 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:30:06.591181 5965 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:30:06.591248 5965 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 06:30:06.591361 5965 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 06:30:06.591418 5965 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260811 06:30:06.601756 5965 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:36579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786429755927508-29631-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786429755927508-29631-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.239.254:45073
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:45073
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.239.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786429755927508-29631-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786429755927508-29631-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:30:06.603210 5965 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:30:06.604671 5965 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:30:06.617460 5975 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:30:06.617496 5982 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:30:06.617555 5976 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:30:06.618317 5965 server_base.cc:1034] running on GCE node
I20260811 06:30:06.619052 5965 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:30:06.620447 5965 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:30:06.621657 5965 hybrid_clock.cc:648] HybridClock initialized: now 1786429806621601 us; error 43 us; skew 500 ppm
I20260811 06:30:06.622000 5965 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:30:06.624449 5965 webserver.cc:460] Webserver started at http://127.28.239.254:35633/ using document root <none> and password file <none>
I20260811 06:30:06.625186 5965 fs_manager.cc:359] Metadata directory not provided
I20260811 06:30:06.625293 5965 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:30:06.625639 5965 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:30:06.628592 5965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786429755927508-29631-0/minicluster-data/master-0/data/instance:
uuid: "148e35c086cd4b079501d3e4071450ca"
format_stamp: "Formatted at 2026-08-11 06:30:06 on dist-test-slave-gkw7"
server_key: "117f2847fde29c5b736abae13c6c0c47"
server_key_iv: "82a9919ce24368cece041445333db85c"
server_key_version: "encryptionkey@0"
I20260811 06:30:06.629384 5965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786429755927508-29631-0/minicluster-data/master-0/wal/instance:
uuid: "148e35c086cd4b079501d3e4071450ca"
format_stamp: "Formatted at 2026-08-11 06:30:06 on dist-test-slave-gkw7"
server_key: "117f2847fde29c5b736abae13c6c0c47"
server_key_iv: "82a9919ce24368cece041445333db85c"
server_key_version: "encryptionkey@0"
I20260811 06:30:06.634807 5965 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 06:30:06.638341 5989 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:06.639810 5965 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 06:30:06.640017 5965 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786429755927508-29631-0/minicluster-data/master-0/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786429755927508-29631-0/minicluster-data/master-0/wal
uuid: "148e35c086cd4b079501d3e4071450ca"
format_stamp: "Formatted at 2026-08-11 06:30:06 on dist-test-slave-gkw7"
server_key: "117f2847fde29c5b736abae13c6c0c47"
server_key_iv: "82a9919ce24368cece041445333db85c"
server_key_version: "encryptionkey@0"
I20260811 06:30:06.640252 5965 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786429755927508-29631-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786429755927508-29631-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786429755927508-29631-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:30:06.657673 5965 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:30:06.658596 5965 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:30:06.658916 5965 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:30:06.678930 5965 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.254:45073
I20260811 06:30:06.678961 6068 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.254:45073 every 8 connection(s)
I20260811 06:30:06.680454 5965 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
I20260811 06:30:06.684553 6069 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:30:06.686085 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 5965
I20260811 06:30:06.686331 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786429755927508-29631-0/minicluster-data/master-0/wal/instance
I20260811 06:30:06.687008 29631 external_mini_cluster.cc:1428] Setting key 3b55026dd7c8b671594090cb1646266d
I20260811 06:30:06.697119 6069 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 148e35c086cd4b079501d3e4071450ca: Bootstrap starting.
I20260811 06:30:06.701066 6069 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 148e35c086cd4b079501d3e4071450ca: Neither blocks nor log segments found. Creating new log.
I20260811 06:30:06.701997 6074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:06.689117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34788 (local address 127.28.239.254:45073)
0811 06:30:06.690267 (+ 1150us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:06.690282 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:06.690332 (+ 50us) server_negotiation.cc:407] Connection header received
0811 06:30:06.692039 (+ 1707us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:06.692063 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:06.692563 (+ 500us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:06.692905 (+ 342us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:06.693661 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:06.695082 (+ 1421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:06.696907 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:06.697840 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:06.697979 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:06.699340 (+ 1361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:06.699347 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:06.699362 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:30:06.699927 (+ 565us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:06.700023 (+ 96us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:30:06.700046 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:06.700128 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:06.700414 (+ 286us) server_negotiation.cc:299] Negotiation successful
0811 06:30:06.700780 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":691,"thread_start_us":451,"threads_started":1}
I20260811 06:30:06.702479 6069 log.cc:824] T 00000000000000000000000000000000 P 148e35c086cd4b079501d3e4071450ca: Log is configured to *not* fsync() on all Append() calls
I20260811 06:30:06.705843 6069 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 148e35c086cd4b079501d3e4071450ca: No bootstrap required, opened a new log
I20260811 06:30:06.706254 29631 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 06:30:06.714380 6069 raft_consensus.cc:348] T 00000000000000000000000000000000 P 148e35c086cd4b079501d3e4071450ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "148e35c086cd4b079501d3e4071450ca" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 45073 } }
I20260811 06:30:06.714762 6069 raft_consensus.cc:374] T 00000000000000000000000000000000 P 148e35c086cd4b079501d3e4071450ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:30:06.714872 6069 raft_consensus.cc:729] T 00000000000000000000000000000000 P 148e35c086cd4b079501d3e4071450ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 148e35c086cd4b079501d3e4071450ca, State: Initialized, Role: FOLLOWER
I20260811 06:30:06.715442 6069 consensus_queue.cc:260] T 00000000000000000000000000000000 P 148e35c086cd4b079501d3e4071450ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "148e35c086cd4b079501d3e4071450ca" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 45073 } }
I20260811 06:30:06.715669 6069 raft_consensus.cc:388] T 00000000000000000000000000000000 P 148e35c086cd4b079501d3e4071450ca [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:30:06.715822 6069 raft_consensus.cc:482] T 00000000000000000000000000000000 P 148e35c086cd4b079501d3e4071450ca [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:30:06.715991 6069 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 148e35c086cd4b079501d3e4071450ca [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:30:06.718348 6069 raft_consensus.cc:504] T 00000000000000000000000000000000 P 148e35c086cd4b079501d3e4071450ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "148e35c086cd4b079501d3e4071450ca" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 45073 } }
I20260811 06:30:06.718945 6069 leader_election.cc:302] T 00000000000000000000000000000000 P 148e35c086cd4b079501d3e4071450ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 148e35c086cd4b079501d3e4071450ca; no voters:
I20260811 06:30:06.719552 6069 leader_election.cc:288] T 00000000000000000000000000000000 P 148e35c086cd4b079501d3e4071450ca [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:30:06.719726 6078 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 148e35c086cd4b079501d3e4071450ca [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:30:06.720475 6078 raft_consensus.cc:686] T 00000000000000000000000000000000 P 148e35c086cd4b079501d3e4071450ca [term 1 LEADER]: Becoming Leader. State: Replica: 148e35c086cd4b079501d3e4071450ca, State: Running, Role: LEADER
I20260811 06:30:06.721007 6078 consensus_queue.cc:237] T 00000000000000000000000000000000 P 148e35c086cd4b079501d3e4071450ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "148e35c086cd4b079501d3e4071450ca" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 45073 } }
I20260811 06:30:06.721767 6069 sys_catalog.cc:564] T 00000000000000000000000000000000 P 148e35c086cd4b079501d3e4071450ca [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:30:06.724342 6082 sys_catalog.cc:455] T 00000000000000000000000000000000 P 148e35c086cd4b079501d3e4071450ca [sys.catalog]: SysCatalogTable state changed. Reason: New leader 148e35c086cd4b079501d3e4071450ca. Latest consensus state: current_term: 1 leader_uuid: "148e35c086cd4b079501d3e4071450ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "148e35c086cd4b079501d3e4071450ca" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 45073 } } }
I20260811 06:30:06.724526 6079 sys_catalog.cc:455] T 00000000000000000000000000000000 P 148e35c086cd4b079501d3e4071450ca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "148e35c086cd4b079501d3e4071450ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "148e35c086cd4b079501d3e4071450ca" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 45073 } } }
I20260811 06:30:06.724790 6082 sys_catalog.cc:458] T 00000000000000000000000000000000 P 148e35c086cd4b079501d3e4071450ca [sys.catalog]: This master's current role is: LEADER
I20260811 06:30:06.724967 6079 sys_catalog.cc:458] T 00000000000000000000000000000000 P 148e35c086cd4b079501d3e4071450ca [sys.catalog]: This master's current role is: LEADER
I20260811 06:30:06.726948 6086 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:30:06.732270 6086 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:30:06.742432 6086 catalog_manager.cc:1347] Generated new cluster ID: 719af428c0024119ba165c172d733fe0
I20260811 06:30:06.742564 6086 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:30:06.752475 6086 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:30:06.754283 6086 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:30:06.771135 6086 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 148e35c086cd4b079501d3e4071450ca: Generated new TSK 0
I20260811 06:30:06.772029 6086 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:30:07.301923 6074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:06.793776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34798 (local address 127.28.239.254:45073)
0811 06:30:06.793987 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:06.793994 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:06.794535 (+ 541us) server_negotiation.cc:407] Connection header received
0811 06:30:06.794733 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:06.794740 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:06.794845 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:06.795020 (+ 175us) server_negotiation.cc:226] Negotiated authn=JWT
0811 06:30:06.795464 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:06.796685 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:06.798169 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:06.799019 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:06.799160 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:06.799646 (+ 486us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 06:30:07.301160 (+501514us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 06:30:07.301647 (+ 487us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:34798: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":71}
W20260811 06:30:07.302043 6074 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:34798: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260811 06:30:07.302301 6114 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:30:06.793751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:45073 (local address 127.0.0.1:34798)
0811 06:30:06.794427 (+ 676us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:06.794447 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:06.794571 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:06.795046 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:06.795049 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:06.795059 (+ 10us) client_negotiation.cc:189] Negotiated authn=JWT
0811 06:30:06.795306 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:06.795314 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:06.796844 (+ 1530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:06.796848 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:06.798021 (+ 1173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:06.798034 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:06.799191 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:06.799196 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:06.799330 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:06.799362 (+ 32us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 06:30:07.301496 (+502134us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 06:30:07.301869 (+ 373us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.28.239.254:45073: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":357,"threads_started":1}
I20260811 06:30:07.303848 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 5965
I20260811 06:30:07.305184 6064 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:30:07.411449 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 5965
2026-08-11T06:30:07Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (1273 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (440 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (1629 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:30:09 dist-test-slave-gkw7 krb5kdc[6143](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:30:09 dist-test-slave-gkw7 krb5kdc[6143](info): set up 2 sockets
Aug 11 06:30:09 dist-test-slave-gkw7 krb5kdc[6143](info): commencing operation
krb5kdc: starting...
W20260811 06:30:10.813959 29631 mini_kdc.cc:121] Time spent starting KDC: real 1.284s user 0.000s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/krb5kdc/test-user.keytab.
Aug 11 06:30:10 dist-test-slave-gkw7 krb5kdc[6143](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429810, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:30:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:30:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.239.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.239.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.239.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab.
Entry for principal kudu/127.28.239.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab.
I20260811 06:30:11.069422 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:39201
--webserver_interface=127.28.239.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab
--principal=kudu/127.28.239.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:34685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.239.254:39201
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:30:11.435741 6162 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:30:11.436223 6162 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:30:11.436455 6162 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:30:11.446120 6162 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:30:11.446282 6162 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:30:11.446388 6162 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:30:11.446496 6162 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:30:11.446578 6162 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:30:11.456907 6162 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:34685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.239.254:39201
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab
--principal=kudu/127.28.239.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:39201
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.239.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:30:11.458367 6162 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:30:11.459719 6162 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:30:11.472189 6173 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:30:11.472160 6169 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:30:11.472141 6171 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:30:11.473073 6162 server_base.cc:1034] running on GCE node
I20260811 06:30:11.474042 6162 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:30:11.475499 6162 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:30:11.476809 6162 hybrid_clock.cc:648] HybridClock initialized: now 1786429811476765 us; error 52 us; skew 500 ppm
I20260811 06:30:11.477136 6162 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:30:11 dist-test-slave-gkw7 krb5kdc[6143](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429811, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:30:11.483800 6162 init.cc:377] Logged in from keytab as kudu/127.28.239.254@KRBTEST.COM (short username kudu)
I20260811 06:30:11.485854 6162 webserver.cc:460] Webserver started at http://127.28.239.254:33571/ using document root <none> and password file <none>
I20260811 06:30:11.486627 6162 fs_manager.cc:359] Metadata directory not provided
I20260811 06:30:11.486785 6162 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:30:11.487144 6162 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:30:11.490394 6162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/minicluster-data/master-0/data/instance:
uuid: "d2f08cdd7bde47f39d4571d45a3c3484"
format_stamp: "Formatted at 2026-08-11 06:30:11 on dist-test-slave-gkw7"
server_key: "d57e7d00c670c5f96a70605d7a89eec1"
server_key_iv: "902569a3af07fea19575b86a26d0919a"
server_key_version: "encryptionkey@0"
I20260811 06:30:11.491232 6162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/minicluster-data/master-0/wal/instance:
uuid: "d2f08cdd7bde47f39d4571d45a3c3484"
format_stamp: "Formatted at 2026-08-11 06:30:11 on dist-test-slave-gkw7"
server_key: "d57e7d00c670c5f96a70605d7a89eec1"
server_key_iv: "902569a3af07fea19575b86a26d0919a"
server_key_version: "encryptionkey@0"
I20260811 06:30:11.496907 6162 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 06:30:11.500686 6182 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:11.502182 6162 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:30:11.502377 6162 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/minicluster-data/master-0/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/minicluster-data/master-0/wal
uuid: "d2f08cdd7bde47f39d4571d45a3c3484"
format_stamp: "Formatted at 2026-08-11 06:30:11 on dist-test-slave-gkw7"
server_key: "d57e7d00c670c5f96a70605d7a89eec1"
server_key_iv: "902569a3af07fea19575b86a26d0919a"
server_key_version: "encryptionkey@0"
I20260811 06:30:11.502591 6162 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:30:11.515086 6162 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:30:11.519119 6162 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:30:11.519443 6162 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:30:11.538800 6162 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.254:39201
I20260811 06:30:11.538826 6262 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.254:39201 every 8 connection(s)
I20260811 06:30:11.540299 6162 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
I20260811 06:30:11.544243 6263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:30:11.547330 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 6162
I20260811 06:30:11.547577 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/minicluster-data/master-0/wal/instance
I20260811 06:30:11.548403 29631 external_mini_cluster.cc:1428] Setting key ff54572aec5aefd3405a4a7750a3c4eb
I20260811 06:30:11.557588 6263 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2f08cdd7bde47f39d4571d45a3c3484: Bootstrap starting.
I20260811 06:30:11.561686 6263 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d2f08cdd7bde47f39d4571d45a3c3484: Neither blocks nor log segments found. Creating new log.
I20260811 06:30:11.563189 6263 log.cc:824] T 00000000000000000000000000000000 P d2f08cdd7bde47f39d4571d45a3c3484: Log is configured to *not* fsync() on all Append() calls
Aug 11 06:30:11 dist-test-slave-gkw7 krb5kdc[6143](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429810, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:30:11.566949 6263 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2f08cdd7bde47f39d4571d45a3c3484: No bootstrap required, opened a new log
I20260811 06:30:11.574607 6267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:11.550646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45398 (local address 127.28.239.254:39201)
0811 06:30:11.551541 (+ 895us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:11.551553 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:11.551595 (+ 42us) server_negotiation.cc:407] Connection header received
0811 06:30:11.553286 (+ 1691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:11.553311 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:11.553809 (+ 498us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:11.554156 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:11.556021 (+ 1865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:11.557398 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:11.559217 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:11.559890 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:11.560030 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:11.565258 (+ 5228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:11.565279 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:11.565299 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:30:11.565347 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:11.569483 (+ 4136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:11.570283 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:11.570293 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:11.570302 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:11.570554 (+ 252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:11.570966 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:11.570971 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:11.570974 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:11.571779 (+ 805us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:11.572113 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:11.572494 (+ 381us) server_negotiation.cc:299] Negotiation successful
0811 06:30:11.573011 (+ 517us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":694,"thread_start_us":470,"threads_started":1}
I20260811 06:30:11.577749 6263 raft_consensus.cc:348] T 00000000000000000000000000000000 P d2f08cdd7bde47f39d4571d45a3c3484 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2f08cdd7bde47f39d4571d45a3c3484" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 39201 } }
I20260811 06:30:11.578161 6263 raft_consensus.cc:374] T 00000000000000000000000000000000 P d2f08cdd7bde47f39d4571d45a3c3484 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:30:11.578289 6263 raft_consensus.cc:729] T 00000000000000000000000000000000 P d2f08cdd7bde47f39d4571d45a3c3484 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2f08cdd7bde47f39d4571d45a3c3484, State: Initialized, Role: FOLLOWER
I20260811 06:30:11.578958 6263 consensus_queue.cc:260] T 00000000000000000000000000000000 P d2f08cdd7bde47f39d4571d45a3c3484 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2f08cdd7bde47f39d4571d45a3c3484" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 39201 } }
I20260811 06:30:11.579213 6263 raft_consensus.cc:388] T 00000000000000000000000000000000 P d2f08cdd7bde47f39d4571d45a3c3484 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:30:11.579370 6263 raft_consensus.cc:482] T 00000000000000000000000000000000 P d2f08cdd7bde47f39d4571d45a3c3484 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:30:11.579562 6263 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d2f08cdd7bde47f39d4571d45a3c3484 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:30:11.581863 6263 raft_consensus.cc:504] T 00000000000000000000000000000000 P d2f08cdd7bde47f39d4571d45a3c3484 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2f08cdd7bde47f39d4571d45a3c3484" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 39201 } }
I20260811 06:30:11.582388 6263 leader_election.cc:302] T 00000000000000000000000000000000 P d2f08cdd7bde47f39d4571d45a3c3484 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d2f08cdd7bde47f39d4571d45a3c3484; no voters:
I20260811 06:30:11.583070 6263 leader_election.cc:288] T 00000000000000000000000000000000 P d2f08cdd7bde47f39d4571d45a3c3484 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:30:11.583221 6271 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d2f08cdd7bde47f39d4571d45a3c3484 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:30:11.583954 6271 raft_consensus.cc:686] T 00000000000000000000000000000000 P d2f08cdd7bde47f39d4571d45a3c3484 [term 1 LEADER]: Becoming Leader. State: Replica: d2f08cdd7bde47f39d4571d45a3c3484, State: Running, Role: LEADER
I20260811 06:30:11.584483 6271 consensus_queue.cc:237] T 00000000000000000000000000000000 P d2f08cdd7bde47f39d4571d45a3c3484 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2f08cdd7bde47f39d4571d45a3c3484" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 39201 } }
I20260811 06:30:11.585251 6263 sys_catalog.cc:564] T 00000000000000000000000000000000 P d2f08cdd7bde47f39d4571d45a3c3484 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:30:11.587433 6273 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2f08cdd7bde47f39d4571d45a3c3484 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d2f08cdd7bde47f39d4571d45a3c3484. Latest consensus state: current_term: 1 leader_uuid: "d2f08cdd7bde47f39d4571d45a3c3484" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2f08cdd7bde47f39d4571d45a3c3484" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 39201 } } }
I20260811 06:30:11.587808 6273 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2f08cdd7bde47f39d4571d45a3c3484 [sys.catalog]: This master's current role is: LEADER
I20260811 06:30:11.588744 6280 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:30:11.589711 6272 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2f08cdd7bde47f39d4571d45a3c3484 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d2f08cdd7bde47f39d4571d45a3c3484" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2f08cdd7bde47f39d4571d45a3c3484" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 39201 } } }
I20260811 06:30:11.590087 6272 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2f08cdd7bde47f39d4571d45a3c3484 [sys.catalog]: This master's current role is: LEADER
I20260811 06:30:11.594633 6280 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:30:11.604799 6280 catalog_manager.cc:1347] Generated new cluster ID: 5c36bfe79df04179bdac04b8c37aa2b7
I20260811 06:30:11.604948 6280 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:30:11.617457 6280 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:30:11.618857 6280 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:30:11.630374 6280 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d2f08cdd7bde47f39d4571d45a3c3484: Generated new TSK 0
I20260811 06:30:11.631249 6280 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.239.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.239.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.239.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.193.keytab.
Entry for principal kudu/127.28.239.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.193.keytab.
I20260811 06:30:11.706435 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.193:0
--local_ip_for_outbound_sockets=127.28.239.193
--webserver_interface=127.28.239.193
--webserver_port=0
--tserver_master_addrs=127.28.239.254:39201
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.193.keytab
--principal=kudu/127.28.239.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:34685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:30:12.093401 6308 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:30:12.093725 6308 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:30:12.093890 6308 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:30:12.101720 6308 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:30:12.101861 6308 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:30:12.101979 6308 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.193
I20260811 06:30:12.112161 6308 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:34685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.193.keytab
--principal=kudu/127.28.239.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.239.193
--webserver_port=0
--tserver_master_addrs=127.28.239.254:39201
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.193
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:30:12.113510 6308 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:30:12.114935 6308 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:30:12.128844 6314 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:30:12.128945 6318 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:30:12.128845 6315 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:30:12.130946 6308 server_base.cc:1034] running on GCE node
I20260811 06:30:12.131651 6308 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:30:12.132747 6308 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:30:12.133994 6308 hybrid_clock.cc:648] HybridClock initialized: now 1786429812133927 us; error 59 us; skew 500 ppm
I20260811 06:30:12.134366 6308 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:30:12 dist-test-slave-gkw7 krb5kdc[6143](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429812, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:30:12.137744 6308 init.cc:377] Logged in from keytab as kudu/127.28.239.193@KRBTEST.COM (short username kudu)
I20260811 06:30:12.139293 6308 webserver.cc:460] Webserver started at http://127.28.239.193:41335/ using document root <none> and password file <none>
I20260811 06:30:12.140062 6308 fs_manager.cc:359] Metadata directory not provided
I20260811 06:30:12.140213 6308 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:30:12.140538 6308 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:30:12.143661 6308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/minicluster-data/ts-0/data/instance:
uuid: "c08f9361cb5b496dabad4b134c3b99b1"
format_stamp: "Formatted at 2026-08-11 06:30:12 on dist-test-slave-gkw7"
server_key: "905bd49c2549f6b73c91bcd43f3ab7a3"
server_key_iv: "eadfc6ea7ec68894779ded7c792cbc8f"
server_key_version: "encryptionkey@0"
I20260811 06:30:12.144490 6308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/minicluster-data/ts-0/wal/instance:
uuid: "c08f9361cb5b496dabad4b134c3b99b1"
format_stamp: "Formatted at 2026-08-11 06:30:12 on dist-test-slave-gkw7"
server_key: "905bd49c2549f6b73c91bcd43f3ab7a3"
server_key_iv: "eadfc6ea7ec68894779ded7c792cbc8f"
server_key_version: "encryptionkey@0"
I20260811 06:30:12.150121 6308 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 06:30:12.153833 6329 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:12.155306 6308 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 06:30:12.155525 6308 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/minicluster-data/ts-0/wal
uuid: "c08f9361cb5b496dabad4b134c3b99b1"
format_stamp: "Formatted at 2026-08-11 06:30:12 on dist-test-slave-gkw7"
server_key: "905bd49c2549f6b73c91bcd43f3ab7a3"
server_key_iv: "eadfc6ea7ec68894779ded7c792cbc8f"
server_key_version: "encryptionkey@0"
I20260811 06:30:12.155766 6308 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:30:12.172281 6308 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:30:12.176726 6308 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:30:12.177045 6308 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:30:12.178138 6308 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:30:12.179575 6308 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:30:12.179701 6308 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:12.179867 6308 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:30:12.179949 6308 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:12.215816 6308 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.193:40691
I20260811 06:30:12.215857 6517 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.193:40691 every 8 connection(s)
I20260811 06:30:12.217252 6308 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
I20260811 06:30:12.217836 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 6308
I20260811 06:30:12.218060 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/minicluster-data/ts-0/wal/instance
I20260811 06:30:12.218941 29631 external_mini_cluster.cc:1428] Setting key ba71feb60f63dc9d16bb96fe15109d89
Aug 11 06:30:12 dist-test-slave-gkw7 krb5kdc[6143](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429812, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.193@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:30:12.241074 6522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:12.220268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.193:52463 (local address 127.28.239.254:39201)
0811 06:30:12.220795 (+ 527us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:12.220801 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:12.222008 (+ 1207us) server_negotiation.cc:407] Connection header received
0811 06:30:12.225041 (+ 3033us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:12.225045 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:12.225146 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:12.225256 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:12.227490 (+ 2234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:12.228234 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:12.230074 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:12.230526 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:12.230680 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:12.235017 (+ 4337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:12.235022 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:12.235025 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:30:12.235059 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:12.237816 (+ 2757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:12.238601 (+ 785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:12.238606 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:12.238608 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:12.238734 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:12.239210 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:12.239214 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:12.239216 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:12.239510 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:12.239678 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:12.240449 (+ 771us) server_negotiation.cc:299] Negotiation successful
0811 06:30:12.240693 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":245,"threads_started":1}
I20260811 06:30:12.242198 6530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:12.220915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:39201 (local address 127.28.239.193:52463)
0811 06:30:12.221811 (+ 896us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:12.221875 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:12.224718 (+ 2843us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:12.225568 (+ 850us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:12.225583 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:12.226410 (+ 827us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:12.227277 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:12.227305 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:12.228439 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:12.228443 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:12.229908 (+ 1465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:12.229921 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:12.230787 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:12.230798 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:12.231161 (+ 363us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:12.231971 (+ 810us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:30:12.232014 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:12.234752 (+ 2738us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:30:12.237967 (+ 3215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:30:12.237977 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:30:12.237996 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:30:12.238457 (+ 461us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:30:12.238855 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:30:12.238859 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:30:12.238863 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:30:12.239072 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:30:12.239653 (+ 581us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:30:12.239663 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:30:12.240036 (+ 373us) client_negotiation.cc:769] Sending connection context
0811 06:30:12.240321 (+ 285us) client_negotiation.cc:240] Negotiation successful
0811 06:30:12.240919 (+ 598us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":543,"thread_start_us":316,"threads_started":1}
I20260811 06:30:12.244045 6518 heartbeater.cc:347] Connected to a master server at 127.28.239.254:39201
I20260811 06:30:12.244371 6518 heartbeater.cc:464] Registering TS with master...
I20260811 06:30:12.245151 6518 heartbeater.cc:511] Master 127.28.239.254:39201 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.28.239.194@KRBTEST.COM; defaulting to no policy
I20260811 06:30:12.247624 6209 ts_manager.cc:194] Registered new tserver with Master: c08f9361cb5b496dabad4b134c3b99b1 (127.28.239.193:40691)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.239.194@KRBTEST.COM" created.
I20260811 06:30:12.249287 6209 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.239.193@KRBTEST.COM'} at 127.28.239.193:52463
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.239.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.194.keytab.
Entry for principal kudu/127.28.239.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.194.keytab.
I20260811 06:30:12.280260 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.194:0
--local_ip_for_outbound_sockets=127.28.239.194
--webserver_interface=127.28.239.194
--webserver_port=0
--tserver_master_addrs=127.28.239.254:39201
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.194.keytab
--principal=kudu/127.28.239.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:34685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:30:12.634769 6534 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:30:12.635062 6534 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:30:12.635159 6534 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:30:12.642321 6534 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:30:12.642446 6534 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:30:12.642560 6534 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.194
I20260811 06:30:12.652311 6534 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:34685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.194.keytab
--principal=kudu/127.28.239.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.239.194
--webserver_port=0
--tserver_master_addrs=127.28.239.254:39201
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.194
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:30:12.653851 6534 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:30:12.655313 6534 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:30:12.669548 6540 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:30:12.669497 6541 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:30:12.669821 6543 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:30:12.670857 6534 server_base.cc:1034] running on GCE node
I20260811 06:30:12.671527 6534 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:30:12.672555 6534 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:30:12.673797 6534 hybrid_clock.cc:648] HybridClock initialized: now 1786429812673747 us; error 46 us; skew 500 ppm
I20260811 06:30:12.674162 6534 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:30:12 dist-test-slave-gkw7 krb5kdc[6143](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429812, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:30:12.677498 6534 init.cc:377] Logged in from keytab as kudu/127.28.239.194@KRBTEST.COM (short username kudu)
I20260811 06:30:12.679131 6534 webserver.cc:460] Webserver started at http://127.28.239.194:34697/ using document root <none> and password file <none>
I20260811 06:30:12.679878 6534 fs_manager.cc:359] Metadata directory not provided
I20260811 06:30:12.680034 6534 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:30:12.680359 6534 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:30:12.683423 6534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/minicluster-data/ts-1/data/instance:
uuid: "943242f84df84b1b9102d3b899c7a7e3"
format_stamp: "Formatted at 2026-08-11 06:30:12 on dist-test-slave-gkw7"
server_key: "494e28fe8fc20008c5802b0c6cf22173"
server_key_iv: "d3adb5fcdc62c354c4ac08ff1e2c72e8"
server_key_version: "encryptionkey@0"
I20260811 06:30:12.684271 6534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/minicluster-data/ts-1/wal/instance:
uuid: "943242f84df84b1b9102d3b899c7a7e3"
format_stamp: "Formatted at 2026-08-11 06:30:12 on dist-test-slave-gkw7"
server_key: "494e28fe8fc20008c5802b0c6cf22173"
server_key_iv: "d3adb5fcdc62c354c4ac08ff1e2c72e8"
server_key_version: "encryptionkey@0"
I20260811 06:30:12.689663 6534 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260811 06:30:12.693334 6553 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:12.694820 6534 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 06:30:12.695011 6534 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/minicluster-data/ts-1/wal
uuid: "943242f84df84b1b9102d3b899c7a7e3"
format_stamp: "Formatted at 2026-08-11 06:30:12 on dist-test-slave-gkw7"
server_key: "494e28fe8fc20008c5802b0c6cf22173"
server_key_iv: "d3adb5fcdc62c354c4ac08ff1e2c72e8"
server_key_version: "encryptionkey@0"
I20260811 06:30:12.695258 6534 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:30:12.714300 6534 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:30:12.718479 6534 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:30:12.718742 6534 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:30:12.719695 6534 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:30:12.721071 6534 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:30:12.721149 6534 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:12.721238 6534 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:30:12.721284 6534 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:12.755702 6534 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.194:45755
I20260811 06:30:12.755750 6730 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.194:45755 every 8 connection(s)
I20260811 06:30:12.757047 6534 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
I20260811 06:30:12.766736 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 6534
I20260811 06:30:12.766935 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/minicluster-data/ts-1/wal/instance
I20260811 06:30:12.767390 29631 external_mini_cluster.cc:1428] Setting key 636402d4a5e82a22efaa012646d80b59
Aug 11 06:30:12 dist-test-slave-gkw7 krb5kdc[6143](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429812, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.194@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:30:12.778192 6735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:12.759814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.194:53433 (local address 127.28.239.254:39201)
0811 06:30:12.760340 (+ 526us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:12.760347 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:12.761374 (+ 1027us) server_negotiation.cc:407] Connection header received
0811 06:30:12.763221 (+ 1847us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:12.763226 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:12.763302 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:12.763402 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:12.764954 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:12.765776 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:12.767532 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:12.768027 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:12.768178 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:12.772134 (+ 3956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:12.772139 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:12.772142 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:30:12.772176 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:12.774841 (+ 2665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:12.775611 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:12.775619 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:12.775626 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:12.775781 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:12.776260 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:12.776268 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:12.776272 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:12.776626 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:12.776797 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:12.777564 (+ 767us) server_negotiation.cc:299] Negotiation successful
0811 06:30:12.777808 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":383,"thread_start_us":243,"threads_started":1}
I20260811 06:30:12.779593 6736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:12.760451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:39201 (local address 127.28.239.194:53433)
0811 06:30:12.761192 (+ 741us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:12.761242 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:12.762959 (+ 1717us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:12.763629 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:12.763637 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:12.764199 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:12.764792 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:12.764811 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:12.765971 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:12.765975 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:12.767370 (+ 1395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:12.767383 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:12.768333 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:12.768340 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:12.768653 (+ 313us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:12.769571 (+ 918us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:30:12.769609 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:12.771893 (+ 2284us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:30:12.775000 (+ 3107us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:30:12.775009 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:30:12.775026 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:30:12.775483 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:30:12.775902 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:30:12.775907 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:30:12.775923 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:30:12.776138 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:30:12.776761 (+ 623us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:30:12.776770 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:30:12.777136 (+ 366us) client_negotiation.cc:769] Sending connection context
0811 06:30:12.777446 (+ 310us) client_negotiation.cc:240] Negotiation successful
0811 06:30:12.778038 (+ 592us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":292,"threads_started":1}
I20260811 06:30:12.781219 6732 heartbeater.cc:347] Connected to a master server at 127.28.239.254:39201
I20260811 06:30:12.781577 6732 heartbeater.cc:464] Registering TS with master...
I20260811 06:30:12.782518 6732 heartbeater.cc:511] Master 127.28.239.254:39201 requested a full tablet report, sending...
I20260811 06:30:12.784232 6209 ts_manager.cc:194] Registered new tserver with Master: 943242f84df84b1b9102d3b899c7a7e3 (127.28.239.194:45755)
I20260811 06:30:12.785212 6209 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.239.194@KRBTEST.COM'} at 127.28.239.194:53433
WARNING: no policy specified for kudu/127.28.239.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.239.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.239.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.195.keytab.
Entry for principal kudu/127.28.239.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.195.keytab.
I20260811 06:30:12.825145 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.195:0
--local_ip_for_outbound_sockets=127.28.239.195
--webserver_interface=127.28.239.195
--webserver_port=0
--tserver_master_addrs=127.28.239.254:39201
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.195.keytab
--principal=kudu/127.28.239.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:34685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:30:13.170420 6740 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:30:13.170738 6740 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:30:13.170912 6740 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:30:13.178308 6740 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:30:13.178467 6740 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:30:13.178643 6740 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.195
I20260811 06:30:13.188364 6740 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:34685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.195.keytab
--principal=kudu/127.28.239.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.239.195
--webserver_port=0
--tserver_master_addrs=127.28.239.254:39201
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.195
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:30:13.189738 6740 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:30:13.191107 6740 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:30:13.206948 6753 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:30:13.207000 6751 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:30:13.206965 6750 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:30:13.207803 6740 server_base.cc:1034] running on GCE node
I20260811 06:30:13.208400 6740 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:30:13.209296 6740 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:30:13.210494 6740 hybrid_clock.cc:648] HybridClock initialized: now 1786429813210410 us; error 80 us; skew 500 ppm
I20260811 06:30:13.210778 6740 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:30:13 dist-test-slave-gkw7 krb5kdc[6143](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429813, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:30:13.214002 6740 init.cc:377] Logged in from keytab as kudu/127.28.239.195@KRBTEST.COM (short username kudu)
I20260811 06:30:13.215619 6740 webserver.cc:460] Webserver started at http://127.28.239.195:41805/ using document root <none> and password file <none>
I20260811 06:30:13.216509 6740 fs_manager.cc:359] Metadata directory not provided
I20260811 06:30:13.216777 6740 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:30:13.217161 6740 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:30:13.221092 6740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/minicluster-data/ts-2/data/instance:
uuid: "8a16308bb84042a6bae9299481c5f52e"
format_stamp: "Formatted at 2026-08-11 06:30:13 on dist-test-slave-gkw7"
server_key: "c6c2130850ed268565cb3b6065778621"
server_key_iv: "1f201ffeeae5d18f2c1d853f0f94549e"
server_key_version: "encryptionkey@0"
I20260811 06:30:13.222137 6740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/minicluster-data/ts-2/wal/instance:
uuid: "8a16308bb84042a6bae9299481c5f52e"
format_stamp: "Formatted at 2026-08-11 06:30:13 on dist-test-slave-gkw7"
server_key: "c6c2130850ed268565cb3b6065778621"
server_key_iv: "1f201ffeeae5d18f2c1d853f0f94549e"
server_key_version: "encryptionkey@0"
I20260811 06:30:13.227137 6740 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 06:30:13.230567 6770 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:13.231917 6740 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 06:30:13.232051 6740 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/minicluster-data/ts-2/wal
uuid: "8a16308bb84042a6bae9299481c5f52e"
format_stamp: "Formatted at 2026-08-11 06:30:13 on dist-test-slave-gkw7"
server_key: "c6c2130850ed268565cb3b6065778621"
server_key_iv: "1f201ffeeae5d18f2c1d853f0f94549e"
server_key_version: "encryptionkey@0"
I20260811 06:30:13.232214 6740 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:30:13.247689 6740 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:30:13.251639 6740 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:30:13.251895 6740 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:30:13.252462 6518 heartbeater.cc:503] Master 127.28.239.254:39201 was elected leader, sending a full tablet report...
I20260811 06:30:13.252913 6740 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:30:13.254479 6740 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:30:13.254566 6740 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:13.254714 6740 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:30:13.254773 6740 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:13.294471 6740 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.195:43255
I20260811 06:30:13.294509 6945 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.195:43255 every 8 connection(s)
I20260811 06:30:13.295914 6740 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
I20260811 06:30:13.298643 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 6740
I20260811 06:30:13.298827 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/minicluster-data/ts-2/wal/instance
I20260811 06:30:13.299444 29631 external_mini_cluster.cc:1428] Setting key ece839227ac70caf4fe1114a4f5dac0b
Aug 11 06:30:13 dist-test-slave-gkw7 krb5kdc[6143](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429813, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.195@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:30:13.316957 6948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:13.298707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.195:43301 (local address 127.28.239.254:39201)
0811 06:30:13.299334 (+ 627us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:13.299341 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:13.300489 (+ 1148us) server_negotiation.cc:407] Connection header received
0811 06:30:13.303062 (+ 2573us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:13.303067 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:13.303180 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:13.303311 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:13.304812 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:13.305681 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:13.307342 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:13.307744 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:13.307878 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:13.311246 (+ 3368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:13.311251 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:13.311254 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:30:13.311288 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:13.313888 (+ 2600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:13.314827 (+ 939us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:13.314832 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:13.314836 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:13.314980 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:13.315504 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:13.315508 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:13.315510 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:13.315788 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:13.315910 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:13.316522 (+ 612us) server_negotiation.cc:299] Negotiation successful
0811 06:30:13.316695 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":484,"thread_start_us":342,"threads_started":1}
I20260811 06:30:13.317999 6949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:13.299343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:39201 (local address 127.28.239.195:43301)
0811 06:30:13.300313 (+ 970us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:13.300384 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:13.302777 (+ 2393us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:13.303559 (+ 782us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:13.303568 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:13.304071 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:13.304635 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:13.304652 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:13.305842 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:13.305847 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:13.307157 (+ 1310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:13.307171 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:13.307968 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:13.307977 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:13.308214 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:13.308812 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:30:13.308833 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:13.311037 (+ 2204us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:30:13.314018 (+ 2981us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:30:13.314029 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:30:13.314046 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:30:13.314701 (+ 655us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:30:13.315105 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:30:13.315112 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:30:13.315117 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:30:13.315392 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:30:13.315896 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:30:13.315902 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:30:13.316199 (+ 297us) client_negotiation.cc:769] Sending connection context
0811 06:30:13.316430 (+ 231us) client_negotiation.cc:240] Negotiation successful
0811 06:30:13.316813 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":623,"thread_start_us":391,"threads_started":1}
I20260811 06:30:13.319361 6946 heartbeater.cc:347] Connected to a master server at 127.28.239.254:39201
I20260811 06:30:13.319664 6946 heartbeater.cc:464] Registering TS with master...
I20260811 06:30:13.320397 6946 heartbeater.cc:511] Master 127.28.239.254:39201 requested a full tablet report, sending...
I20260811 06:30:13.321939 6209 ts_manager.cc:194] Registered new tserver with Master: 8a16308bb84042a6bae9299481c5f52e (127.28.239.195:43255)
I20260811 06:30:13.322710 6209 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.239.195@KRBTEST.COM'} at 127.28.239.195:43301
I20260811 06:30:13.327533 29631 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/krb5kdc/test-admin.keytab.
Aug 11 06:30:13 dist-test-slave-gkw7 krb5kdc[6143](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429813, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:30:13.360010 29631 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260811 06:30:13.366089 6956 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:30:13.363052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54982 (local address 127.28.239.193:40691)
0811 06:30:13.363939 (+ 887us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:13.363948 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:13.363981 (+ 33us) server_negotiation.cc:407] Connection header received
0811 06:30:13.364066 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:13.364074 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:13.364317 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:13.364491 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:13.365541 (+ 1050us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54982: BlockingRecv error: recv got EOF from 127.0.0.1:54982 (error 108)
Metrics: {"server-negotiator.queue_time_us":726,"thread_start_us":515,"threads_started":1}
W20260811 06:30:13.366911 29631 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/krb5kdc/krb5cc)
Aug 11 06:30:13 dist-test-slave-gkw7 krb5kdc[6143](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429813, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:30:13.368475 29631 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 06:30:13 dist-test-slave-gkw7 krb5kdc[6143](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429813, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:30:13.369920 29631 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 06:30:13 dist-test-slave-gkw7 krb5kdc[6143](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429813, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.239.193@KRBTEST.COM
I20260811 06:30:13.386333 6956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:13.371644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54990 (local address 127.28.239.193:40691)
0811 06:30:13.371844 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:13.371850 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:13.372018 (+ 168us) server_negotiation.cc:407] Connection header received
0811 06:30:13.372217 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:13.372221 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:13.372294 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:13.372405 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:13.373465 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:13.374514 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:13.375775 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:13.376383 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:13.376527 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:13.380437 (+ 3910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:13.380450 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:13.380467 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:30:13.380503 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:13.383394 (+ 2891us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:13.384119 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:13.384127 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:13.384129 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:13.384263 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:13.384653 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:13.384659 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:13.384662 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:13.385054 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:13.385378 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:13.385749 (+ 371us) server_negotiation.cc:299] Negotiation successful
0811 06:30:13.386058 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
W20260811 06:30:13.392189 6956 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:30:13.390493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55006 (local address 127.28.239.193:40691)
0811 06:30:13.390739 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:13.390744 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:13.390951 (+ 207us) server_negotiation.cc:407] Connection header received
0811 06:30:13.391069 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:13.391073 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:13.391142 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:13.391229 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:13.391963 (+ 734us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55006: BlockingRecv error: recv got EOF from 127.0.0.1:55006 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
W20260811 06:30:13.392454 29631 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/krb5kdc/krb5cc' of type FILE
Aug 11 06:30:13 dist-test-slave-gkw7 krb5kdc[6143](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429813, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:30:13.394140 29631 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 06:30:13 dist-test-slave-gkw7 krb5kdc[6143](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429813, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:30:13.395709 29631 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 06:30:13 dist-test-slave-gkw7 krb5kdc[6143](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429813, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.239.193@KRBTEST.COM
I20260811 06:30:13.410410 6956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:13.397052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55014 (local address 127.28.239.193:40691)
0811 06:30:13.397255 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:13.397263 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:13.397553 (+ 290us) server_negotiation.cc:407] Connection header received
0811 06:30:13.397756 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:13.397761 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:13.397819 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:13.397924 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:13.399009 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:13.399800 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:13.401096 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:13.401610 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:13.401748 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:13.405256 (+ 3508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:13.405261 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:13.405264 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:30:13.405322 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:13.407827 (+ 2505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:13.408459 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:13.408471 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:13.408474 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:13.408597 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:13.409009 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:13.409015 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:13.409020 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:13.409302 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:13.409606 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:13.409915 (+ 309us) server_negotiation.cc:299] Negotiation successful
0811 06:30:13.410120 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
W20260811 06:30:13.414651 6956 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:30:13.413166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55018 (local address 127.28.239.193:40691)
0811 06:30:13.413362 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:13.413366 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:13.413430 (+ 64us) server_negotiation.cc:407] Connection header received
0811 06:30:13.413601 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:13.413604 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:13.413670 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:13.413754 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:13.414454 (+ 700us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55018: BlockingRecv error: recv got EOF from 127.0.0.1:55018 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
W20260811 06:30:13.414919 29631 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786429755927508-29631-0/krb5kdc/krb5cc' of type FILE
Aug 11 06:30:13 dist-test-slave-gkw7 krb5kdc[6143](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429813, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:30:13.416416 29631 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 06:30:13 dist-test-slave-gkw7 krb5kdc[6143](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429813, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:30:13.417784 29631 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 06:30:13 dist-test-slave-gkw7 krb5kdc[6143](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429813, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.239.193@KRBTEST.COM
I20260811 06:30:13.431682 6956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:13.419064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55034 (local address 127.28.239.193:40691)
0811 06:30:13.419301 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:13.419308 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:13.419431 (+ 123us) server_negotiation.cc:407] Connection header received
0811 06:30:13.419543 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:13.419547 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:13.419599 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:13.419675 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:13.420523 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:13.421217 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:13.422510 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:13.422937 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:13.423031 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:13.426494 (+ 3463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:13.426499 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:13.426502 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:30:13.426531 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:13.429334 (+ 2803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:13.429973 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:13.429977 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:13.429979 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:13.430128 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:13.430470 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:13.430473 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:13.430475 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:13.430719 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:13.430952 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:13.431137 (+ 185us) server_negotiation.cc:299] Negotiation successful
0811 06:30:13.431375 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 06:30:13.433284 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 6308
I20260811 06:30:13.445715 6956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:13.434082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55050 (local address 127.28.239.193:40691)
0811 06:30:13.434225 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:13.434229 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:13.434596 (+ 367us) server_negotiation.cc:407] Connection header received
0811 06:30:13.434721 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:13.434725 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:13.434779 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:13.434854 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:13.435828 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:13.436494 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:13.437725 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:13.438155 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:13.438252 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:13.440211 (+ 1959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:13.440215 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:13.440218 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:30:13.440243 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:13.442637 (+ 2394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:13.443473 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:13.443483 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:13.443486 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:13.443612 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:13.444091 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:13.444094 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:13.444097 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:13.444344 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:13.444636 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:13.444986 (+ 350us) server_negotiation.cc:299] Negotiation successful
0811 06:30:13.445245 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 06:30:13.446434 6508 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:30:13.531029 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 6308
I20260811 06:30:13.558063 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 6534
Aug 11 06:30:13 dist-test-slave-gkw7 krb5kdc[6143](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429813, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.239.194@KRBTEST.COM
I20260811 06:30:13.573627 6981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:13.559076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41474 (local address 127.28.239.194:45755)
0811 06:30:13.559678 (+ 602us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:13.559685 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:13.559712 (+ 27us) server_negotiation.cc:407] Connection header received
0811 06:30:13.559783 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:13.559790 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:13.560004 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:13.560138 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:13.561118 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:13.562373 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:13.563631 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:13.564207 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:13.564333 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:13.568189 (+ 3856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:13.568201 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:13.568217 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:30:13.568257 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:13.571078 (+ 2821us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:13.571728 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:13.571735 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:13.571737 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:13.571869 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:13.572203 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:13.572206 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:13.572208 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:13.572549 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:13.572771 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:13.573071 (+ 300us) server_negotiation.cc:299] Negotiation successful
0811 06:30:13.573283 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":450,"thread_start_us":341,"threads_started":1}
I20260811 06:30:13.575037 6725 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:30:13.660010 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 6534
I20260811 06:30:13.685336 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 6740
Aug 11 06:30:13 dist-test-slave-gkw7 krb5kdc[6143](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429813, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.239.195@KRBTEST.COM
I20260811 06:30:13.702404 6983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:13.686439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39974 (local address 127.28.239.195:43255)
0811 06:30:13.687193 (+ 754us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:13.687200 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:13.687230 (+ 30us) server_negotiation.cc:407] Connection header received
0811 06:30:13.687292 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:13.687298 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:13.687509 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:13.687695 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:13.688826 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:13.689877 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:13.691623 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:13.692189 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:13.692329 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:13.696538 (+ 4209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:13.696550 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:13.696568 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:30:13.696603 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:13.699722 (+ 3119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:13.700373 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:13.700380 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:13.700382 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:13.700504 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:13.700843 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:13.700846 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:13.700848 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:13.701202 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:13.701444 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:13.701777 (+ 333us) server_negotiation.cc:299] Negotiation successful
0811 06:30:13.702087 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":601,"thread_start_us":429,"threads_started":1}
I20260811 06:30:13.703768 6937 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:30:13.788061 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 6740
I20260811 06:30:13.814797 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 6162
I20260811 06:30:13.816056 6256 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:30:13.923853 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 6162
2026-08-11T06:30:13Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (4447 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:30:14 dist-test-slave-gkw7 krb5kdc[6988](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:30:14 dist-test-slave-gkw7 krb5kdc[6988](info): set up 2 sockets
Aug 11 06:30:14 dist-test-slave-gkw7 krb5kdc[6988](info): commencing operation
krb5kdc: starting...
W20260811 06:30:15.265192 29631 mini_kdc.cc:121] Time spent starting KDC: real 1.288s user 0.000s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/krb5kdc/test-user.keytab.
Aug 11 06:30:15 dist-test-slave-gkw7 krb5kdc[6988](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429815, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:30:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:30:15Z Disabled control of system clock
WARNING: no policy specified for oryx/127.28.239.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.28.239.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.28.239.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/krb5kdc/oryx_127.28.239.254.keytab.
Entry for principal oryx/127.28.239.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/krb5kdc/oryx_127.28.239.254.keytab.
I20260811 06:30:15.533092 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:40129
--webserver_interface=127.28.239.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/krb5kdc/oryx_127.28.239.254.keytab
--principal=oryx/127.28.239.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:43555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.239.254:40129
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:30:15.875473 7004 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:30:15.875844 7004 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:30:15.876004 7004 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:30:15.883469 7004 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:30:15.883644 7004 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:30:15.883754 7004 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:30:15.883844 7004 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:30:15.883910 7004 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:30:15.894189 7004 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:43555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.239.254:40129
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/krb5kdc/oryx_127.28.239.254.keytab
--principal=oryx/127.28.239.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:40129
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.239.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:30:15.895717 7004 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:30:15.897130 7004 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:30:15.909672 7014 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:30:15.909813 7011 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:30:15.909703 7012 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:30:15.910961 7004 server_base.cc:1034] running on GCE node
I20260811 06:30:15.911702 7004 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:30:15.913151 7004 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:30:15.914407 7004 hybrid_clock.cc:648] HybridClock initialized: now 1786429815914354 us; error 75 us; skew 500 ppm
I20260811 06:30:15.914767 7004 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:30:15 dist-test-slave-gkw7 krb5kdc[6988](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429815, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.239.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:30:15.918145 7004 init.cc:377] Logged in from keytab as oryx/127.28.239.254@KRBTEST.COM (short username oryx)
I20260811 06:30:15.919709 7004 webserver.cc:460] Webserver started at http://127.28.239.254:37515/ using document root <none> and password file <none>
I20260811 06:30:15.920431 7004 fs_manager.cc:359] Metadata directory not provided
I20260811 06:30:15.920585 7004 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:30:15.920907 7004 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:30:15.924055 7004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/master-0/data/instance:
uuid: "c1478483ef29416fb130a1d621aafbaa"
format_stamp: "Formatted at 2026-08-11 06:30:15 on dist-test-slave-gkw7"
server_key: "129a89e3d7cce9fe673e6510397afc0c"
server_key_iv: "97f9c06d6b8b1c230c7f56a8bee69bf4"
server_key_version: "encryptionkey@0"
I20260811 06:30:15.924851 7004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/master-0/wal/instance:
uuid: "c1478483ef29416fb130a1d621aafbaa"
format_stamp: "Formatted at 2026-08-11 06:30:15 on dist-test-slave-gkw7"
server_key: "129a89e3d7cce9fe673e6510397afc0c"
server_key_iv: "97f9c06d6b8b1c230c7f56a8bee69bf4"
server_key_version: "encryptionkey@0"
I20260811 06:30:15.930363 7004 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260811 06:30:15.933928 7024 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:15.935245 7004 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 06:30:15.935424 7004 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/master-0/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/master-0/wal
uuid: "c1478483ef29416fb130a1d621aafbaa"
format_stamp: "Formatted at 2026-08-11 06:30:15 on dist-test-slave-gkw7"
server_key: "129a89e3d7cce9fe673e6510397afc0c"
server_key_iv: "97f9c06d6b8b1c230c7f56a8bee69bf4"
server_key_version: "encryptionkey@0"
I20260811 06:30:15.935626 7004 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:30:15.952095 7004 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:30:15.956230 7004 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:30:15.956532 7004 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:30:15.976298 7004 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.254:40129
I20260811 06:30:15.976358 7111 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.254:40129 every 8 connection(s)
I20260811 06:30:15.977828 7004 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
I20260811 06:30:15.979249 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 7004
I20260811 06:30:15.979501 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/master-0/wal/instance
I20260811 06:30:15.980458 29631 external_mini_cluster.cc:1428] Setting key 38b0a3c9fde6c3d44d144f3a1350d626
I20260811 06:30:15.983124 7112 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 06:30:15 dist-test-slave-gkw7 krb5kdc[6988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429815, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM
I20260811 06:30:15.996461 7112 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1478483ef29416fb130a1d621aafbaa: Bootstrap starting.
I20260811 06:30:16.000602 7112 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c1478483ef29416fb130a1d621aafbaa: Neither blocks nor log segments found. Creating new log.
I20260811 06:30:16.002038 7112 log.cc:824] T 00000000000000000000000000000000 P c1478483ef29416fb130a1d621aafbaa: Log is configured to *not* fsync() on all Append() calls
I20260811 06:30:16.005712 7112 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1478483ef29416fb130a1d621aafbaa: No bootstrap required, opened a new log
I20260811 06:30:16.006772 7117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:15.982881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47772 (local address 127.28.239.254:40129)
0811 06:30:15.983744 (+ 863us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:15.983760 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:15.983810 (+ 50us) server_negotiation.cc:407] Connection header received
0811 06:30:15.986174 (+ 2364us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:15.986205 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:15.986731 (+ 526us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:15.987234 (+ 503us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:15.988662 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:15.990782 (+ 2120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:15.992047 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:15.993097 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:15.993302 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:15.997477 (+ 4175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:15.997488 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:15.997506 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:30:15.997584 (+ 78us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:16.001686 (+ 4102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:16.002660 (+ 974us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:16.002669 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:16.002682 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:16.002897 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:16.003401 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:16.003404 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:16.003406 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:16.004122 (+ 716us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:16.004448 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:16.004858 (+ 410us) server_negotiation.cc:299] Negotiation successful
0811 06:30:16.005319 (+ 461us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":627,"thread_start_us":427,"threads_started":1}
I20260811 06:30:16.014489 7112 raft_consensus.cc:348] T 00000000000000000000000000000000 P c1478483ef29416fb130a1d621aafbaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1478483ef29416fb130a1d621aafbaa" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 40129 } }
I20260811 06:30:16.015019 7112 raft_consensus.cc:374] T 00000000000000000000000000000000 P c1478483ef29416fb130a1d621aafbaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:30:16.015132 7112 raft_consensus.cc:729] T 00000000000000000000000000000000 P c1478483ef29416fb130a1d621aafbaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1478483ef29416fb130a1d621aafbaa, State: Initialized, Role: FOLLOWER
I20260811 06:30:16.015789 7112 consensus_queue.cc:260] T 00000000000000000000000000000000 P c1478483ef29416fb130a1d621aafbaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1478483ef29416fb130a1d621aafbaa" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 40129 } }
I20260811 06:30:16.016028 7112 raft_consensus.cc:388] T 00000000000000000000000000000000 P c1478483ef29416fb130a1d621aafbaa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:30:16.016165 7112 raft_consensus.cc:482] T 00000000000000000000000000000000 P c1478483ef29416fb130a1d621aafbaa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:30:16.016316 7112 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c1478483ef29416fb130a1d621aafbaa [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:30:16.018621 7112 raft_consensus.cc:504] T 00000000000000000000000000000000 P c1478483ef29416fb130a1d621aafbaa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1478483ef29416fb130a1d621aafbaa" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 40129 } }
I20260811 06:30:16.019148 7112 leader_election.cc:302] T 00000000000000000000000000000000 P c1478483ef29416fb130a1d621aafbaa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c1478483ef29416fb130a1d621aafbaa; no voters:
I20260811 06:30:16.019778 7112 leader_election.cc:288] T 00000000000000000000000000000000 P c1478483ef29416fb130a1d621aafbaa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:30:16.019945 7120 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c1478483ef29416fb130a1d621aafbaa [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:30:16.020668 7120 raft_consensus.cc:686] T 00000000000000000000000000000000 P c1478483ef29416fb130a1d621aafbaa [term 1 LEADER]: Becoming Leader. State: Replica: c1478483ef29416fb130a1d621aafbaa, State: Running, Role: LEADER
I20260811 06:30:16.021268 7120 consensus_queue.cc:237] T 00000000000000000000000000000000 P c1478483ef29416fb130a1d621aafbaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1478483ef29416fb130a1d621aafbaa" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 40129 } }
I20260811 06:30:16.022086 7112 sys_catalog.cc:564] T 00000000000000000000000000000000 P c1478483ef29416fb130a1d621aafbaa [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:30:16.024623 7121 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1478483ef29416fb130a1d621aafbaa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c1478483ef29416fb130a1d621aafbaa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1478483ef29416fb130a1d621aafbaa" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 40129 } } }
I20260811 06:30:16.024986 7121 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1478483ef29416fb130a1d621aafbaa [sys.catalog]: This master's current role is: LEADER
I20260811 06:30:16.024703 7127 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1478483ef29416fb130a1d621aafbaa [sys.catalog]: SysCatalogTable state changed. Reason: New leader c1478483ef29416fb130a1d621aafbaa. Latest consensus state: current_term: 1 leader_uuid: "c1478483ef29416fb130a1d621aafbaa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1478483ef29416fb130a1d621aafbaa" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 40129 } } }
I20260811 06:30:16.025115 7127 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1478483ef29416fb130a1d621aafbaa [sys.catalog]: This master's current role is: LEADER
I20260811 06:30:16.027385 7131 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:30:16.032815 7131 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:30:16.042474 7131 catalog_manager.cc:1347] Generated new cluster ID: e6f352fda9a24973ae7bc5beafb7d908
I20260811 06:30:16.042596 7131 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:30:16.058337 7131 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:30:16.059875 7131 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:30:16.068638 7131 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c1478483ef29416fb130a1d621aafbaa: Generated new TSK 0
I20260811 06:30:16.069830 7131 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.28.239.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.28.239.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.28.239.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/krb5kdc/oryx_127.28.239.193.keytab.
Entry for principal oryx/127.28.239.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/krb5kdc/oryx_127.28.239.193.keytab.
I20260811 06:30:16.152828 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.193:0
--local_ip_for_outbound_sockets=127.28.239.193
--webserver_interface=127.28.239.193
--webserver_port=0
--tserver_master_addrs=127.28.239.254:40129
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/krb5kdc/oryx_127.28.239.193.keytab
--principal=oryx/127.28.239.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:43555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:30:16.518515 7148 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:30:16.518822 7148 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:30:16.518923 7148 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:30:16.526510 7148 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:30:16.526623 7148 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:30:16.526737 7148 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.193
I20260811 06:30:16.536571 7148 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:43555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/krb5kdc/oryx_127.28.239.193.keytab
--principal=oryx/127.28.239.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.239.193
--webserver_port=0
--tserver_master_addrs=127.28.239.254:40129
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.193
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:30:16.537961 7148 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:30:16.539469 7148 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:30:16.553331 7154 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:30:16.553354 7155 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:30:16.553445 7158 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:30:16.554658 7148 server_base.cc:1034] running on GCE node
I20260811 06:30:16.555393 7148 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:30:16.556469 7148 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:30:16.557711 7148 hybrid_clock.cc:648] HybridClock initialized: now 1786429816557667 us; error 56 us; skew 500 ppm
I20260811 06:30:16.558048 7148 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:30:16 dist-test-slave-gkw7 krb5kdc[6988](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429816, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.239.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:30:16.565267 7148 init.cc:377] Logged in from keytab as oryx/127.28.239.193@KRBTEST.COM (short username oryx)
I20260811 06:30:16.567111 7148 webserver.cc:460] Webserver started at http://127.28.239.193:32959/ using document root <none> and password file <none>
I20260811 06:30:16.568094 7148 fs_manager.cc:359] Metadata directory not provided
I20260811 06:30:16.568311 7148 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:30:16.568670 7148 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:30:16.572089 7148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-0/data/instance:
uuid: "0b83ec152b074f80af2848108d3af254"
format_stamp: "Formatted at 2026-08-11 06:30:16 on dist-test-slave-gkw7"
server_key: "bda6ace95f993a242a367028f6c5508a"
server_key_iv: "e06483c3ac7fa86cc1b08bc828c3a7cd"
server_key_version: "encryptionkey@0"
I20260811 06:30:16.572959 7148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-0/wal/instance:
uuid: "0b83ec152b074f80af2848108d3af254"
format_stamp: "Formatted at 2026-08-11 06:30:16 on dist-test-slave-gkw7"
server_key: "bda6ace95f993a242a367028f6c5508a"
server_key_iv: "e06483c3ac7fa86cc1b08bc828c3a7cd"
server_key_version: "encryptionkey@0"
I20260811 06:30:16.578655 7148 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260811 06:30:16.582736 7169 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:16.584276 7148 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 06:30:16.584491 7148 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-0/wal
uuid: "0b83ec152b074f80af2848108d3af254"
format_stamp: "Formatted at 2026-08-11 06:30:16 on dist-test-slave-gkw7"
server_key: "bda6ace95f993a242a367028f6c5508a"
server_key_iv: "e06483c3ac7fa86cc1b08bc828c3a7cd"
server_key_version: "encryptionkey@0"
I20260811 06:30:16.584724 7148 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:30:16.607540 7148 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:30:16.611909 7148 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:30:16.612226 7148 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:30:16.613323 7148 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:30:16.614811 7148 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:30:16.614926 7148 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:16.615084 7148 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:30:16.615160 7148 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:16.651691 7148 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.193:40357
I20260811 06:30:16.651786 7356 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.193:40357 every 8 connection(s)
I20260811 06:30:16.653083 7148 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
I20260811 06:30:16.655050 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 7148
I20260811 06:30:16.655304 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-0/wal/instance
I20260811 06:30:16.656199 29631 external_mini_cluster.cc:1428] Setting key 978c86c375b3100e001c5a02dcef7aa0
Aug 11 06:30:16 dist-test-slave-gkw7 krb5kdc[6988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429816, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.239.193@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM
I20260811 06:30:16.676904 7359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:16.656344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.193:54361 (local address 127.28.239.254:40129)
0811 06:30:16.656889 (+ 545us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:16.656896 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:16.658076 (+ 1180us) server_negotiation.cc:407] Connection header received
0811 06:30:16.660029 (+ 1953us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:16.660035 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:16.660126 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:16.660263 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:16.661872 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:16.662602 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:16.664225 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:16.664673 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:16.664819 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:16.668736 (+ 3917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:16.668743 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:16.668746 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:30:16.668781 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:16.672953 (+ 4172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:16.673992 (+ 1039us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:16.674002 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:16.674008 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:16.674190 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:16.674675 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:16.674684 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:16.674690 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:16.675120 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:16.675294 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:16.676019 (+ 725us) server_negotiation.cc:299] Negotiation successful
0811 06:30:16.676306 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":399,"thread_start_us":267,"threads_started":1}
I20260811 06:30:16.677682 7360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:16.656924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:40129 (local address 127.28.239.193:54361)
0811 06:30:16.657841 (+ 917us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:16.657897 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:16.659719 (+ 1822us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:16.660513 (+ 794us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:16.660521 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:16.661029 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:16.661690 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:16.661716 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:16.662781 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:16.662785 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:16.664062 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:16.664074 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:16.664910 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:16.664918 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:16.665163 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:16.665934 (+ 771us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:30:16.665957 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:16.668478 (+ 2521us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:30:16.673177 (+ 4699us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:30:16.673188 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:30:16.673207 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:30:16.673840 (+ 633us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:30:16.674319 (+ 479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:30:16.674322 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:30:16.674324 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:30:16.674526 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:30:16.675255 (+ 729us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:30:16.675261 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:30:16.675598 (+ 337us) client_negotiation.cc:769] Sending connection context
0811 06:30:16.675836 (+ 238us) client_negotiation.cc:240] Negotiation successful
0811 06:30:16.676292 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":620,"thread_start_us":362,"threads_started":1}
I20260811 06:30:16.679639 7357 heartbeater.cc:347] Connected to a master server at 127.28.239.254:40129
I20260811 06:30:16.679982 7357 heartbeater.cc:464] Registering TS with master...
I20260811 06:30:16.680764 7357 heartbeater.cc:511] Master 127.28.239.254:40129 requested a full tablet report, sending...
I20260811 06:30:16.683523 7048 ts_manager.cc:194] Registered new tserver with Master: 0b83ec152b074f80af2848108d3af254 (127.28.239.193:40357)
I20260811 06:30:16.686137 7048 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.28.239.193@KRBTEST.COM'} at 127.28.239.193:54361
WARNING: no policy specified for oryx/127.28.239.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.28.239.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.28.239.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/krb5kdc/oryx_127.28.239.194.keytab.
Entry for principal oryx/127.28.239.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/krb5kdc/oryx_127.28.239.194.keytab.
I20260811 06:30:16.721170 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.194:0
--local_ip_for_outbound_sockets=127.28.239.194
--webserver_interface=127.28.239.194
--webserver_port=0
--tserver_master_addrs=127.28.239.254:40129
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/krb5kdc/oryx_127.28.239.194.keytab
--principal=oryx/127.28.239.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:43555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:30:17.083621 7363 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:30:17.083966 7363 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:30:17.084079 7363 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:30:17.092504 7363 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:30:17.092643 7363 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:30:17.092769 7363 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.194
I20260811 06:30:17.104004 7363 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:43555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/krb5kdc/oryx_127.28.239.194.keytab
--principal=oryx/127.28.239.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.239.194
--webserver_port=0
--tserver_master_addrs=127.28.239.254:40129
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.194
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:30:17.105455 7363 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:30:17.107136 7363 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:30:17.122584 7374 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:30:17.122627 7372 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:30:17.122677 7377 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:30:17.123507 7363 server_base.cc:1034] running on GCE node
I20260811 06:30:17.124325 7363 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:30:17.125468 7363 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:30:17.126756 7363 hybrid_clock.cc:648] HybridClock initialized: now 1786429817126703 us; error 41 us; skew 500 ppm
I20260811 06:30:17.127175 7363 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:30:17 dist-test-slave-gkw7 krb5kdc[6988](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429817, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.239.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:30:17.131176 7363 init.cc:377] Logged in from keytab as oryx/127.28.239.194@KRBTEST.COM (short username oryx)
I20260811 06:30:17.132992 7363 webserver.cc:460] Webserver started at http://127.28.239.194:36885/ using document root <none> and password file <none>
I20260811 06:30:17.133908 7363 fs_manager.cc:359] Metadata directory not provided
I20260811 06:30:17.134078 7363 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:30:17.134447 7363 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:30:17.137923 7363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-1/data/instance:
uuid: "b8e3bd4d570d422d8dd77fd3c581232b"
format_stamp: "Formatted at 2026-08-11 06:30:17 on dist-test-slave-gkw7"
server_key: "767e61b7a3898b44f819d61c0d08734f"
server_key_iv: "d65dc91ea550030697e75b28026272de"
server_key_version: "encryptionkey@0"
I20260811 06:30:17.138861 7363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-1/wal/instance:
uuid: "b8e3bd4d570d422d8dd77fd3c581232b"
format_stamp: "Formatted at 2026-08-11 06:30:17 on dist-test-slave-gkw7"
server_key: "767e61b7a3898b44f819d61c0d08734f"
server_key_iv: "d65dc91ea550030697e75b28026272de"
server_key_version: "encryptionkey@0"
I20260811 06:30:17.144941 7363 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260811 06:30:17.149010 7391 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:17.150820 7363 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:30:17.151024 7363 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-1/wal
uuid: "b8e3bd4d570d422d8dd77fd3c581232b"
format_stamp: "Formatted at 2026-08-11 06:30:17 on dist-test-slave-gkw7"
server_key: "767e61b7a3898b44f819d61c0d08734f"
server_key_iv: "d65dc91ea550030697e75b28026272de"
server_key_version: "encryptionkey@0"
I20260811 06:30:17.151302 7363 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:30:17.166592 7363 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:30:17.171329 7363 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:30:17.171660 7363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:30:17.172703 7363 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:30:17.174340 7363 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:30:17.174451 7363 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:17.174611 7363 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:30:17.174696 7363 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:17.213893 7363 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.194:45127
I20260811 06:30:17.213958 7557 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.194:45127 every 8 connection(s)
I20260811 06:30:17.215265 7363 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
I20260811 06:30:17.221908 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 7363
I20260811 06:30:17.222107 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-1/wal/instance
I20260811 06:30:17.222591 29631 external_mini_cluster.cc:1428] Setting key 5c544b9d89a3a16ed233fc3627225965
Aug 11 06:30:17 dist-test-slave-gkw7 krb5kdc[6988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429817, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.239.194@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM
I20260811 06:30:17.240768 7564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:17.218191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.194:41795 (local address 127.28.239.254:40129)
0811 06:30:17.218667 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:17.218674 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:17.219739 (+ 1065us) server_negotiation.cc:407] Connection header received
0811 06:30:17.221617 (+ 1878us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:17.221624 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:17.221739 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:17.221894 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:17.223905 (+ 2011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:17.224960 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:17.227632 (+ 2672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:17.228286 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:17.228403 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:17.232764 (+ 4361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:17.232774 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:17.232782 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:30:17.232833 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:17.236505 (+ 3672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:17.237669 (+ 1164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:17.237674 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:17.237677 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:17.237830 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:17.238439 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:17.238443 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:17.238446 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:17.238877 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:17.239069 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:17.240214 (+ 1145us) server_negotiation.cc:299] Negotiation successful
0811 06:30:17.240440 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":346,"thread_start_us":218,"threads_started":1}
I20260811 06:30:17.242440 7565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:17.218785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:40129 (local address 127.28.239.194:41795)
0811 06:30:17.219574 (+ 789us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:17.219618 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:17.221290 (+ 1672us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:17.222159 (+ 869us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:17.222174 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:17.222965 (+ 791us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:17.223731 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:17.223753 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:17.225124 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:17.225129 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:17.227435 (+ 2306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:17.227453 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:17.228517 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:17.228527 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:17.228873 (+ 346us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:17.229678 (+ 805us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:30:17.229713 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:17.232485 (+ 2772us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:30:17.236724 (+ 4239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:30:17.236736 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:30:17.236758 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:30:17.237475 (+ 717us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:30:17.237961 (+ 486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:30:17.237968 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:30:17.237975 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:30:17.238270 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:30:17.239056 (+ 786us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:30:17.239068 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:30:17.239604 (+ 536us) client_negotiation.cc:769] Sending connection context
0811 06:30:17.240014 (+ 410us) client_negotiation.cc:240] Negotiation successful
0811 06:30:17.240700 (+ 686us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":330,"threads_started":1}
I20260811 06:30:17.244144 7559 heartbeater.cc:347] Connected to a master server at 127.28.239.254:40129
I20260811 06:30:17.244485 7559 heartbeater.cc:464] Registering TS with master...
I20260811 06:30:17.245286 7559 heartbeater.cc:511] Master 127.28.239.254:40129 requested a full tablet report, sending...
I20260811 06:30:17.246939 7048 ts_manager.cc:194] Registered new tserver with Master: b8e3bd4d570d422d8dd77fd3c581232b (127.28.239.194:45127)
I20260811 06:30:17.247843 7048 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.28.239.194@KRBTEST.COM'} at 127.28.239.194:41795
WARNING: no policy specified for oryx/127.28.239.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.28.239.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.28.239.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/krb5kdc/oryx_127.28.239.195.keytab.
Entry for principal oryx/127.28.239.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/krb5kdc/oryx_127.28.239.195.keytab.
I20260811 06:30:17.289819 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.195:0
--local_ip_for_outbound_sockets=127.28.239.195
--webserver_interface=127.28.239.195
--webserver_port=0
--tserver_master_addrs=127.28.239.254:40129
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/krb5kdc/oryx_127.28.239.195.keytab
--principal=oryx/127.28.239.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:43555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:30:17.650997 7568 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:30:17.651432 7568 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:30:17.651608 7568 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:30:17.659489 7568 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:30:17.659667 7568 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:30:17.659847 7568 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.195
I20260811 06:30:17.669909 7568 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:43555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/krb5kdc/oryx_127.28.239.195.keytab
--principal=oryx/127.28.239.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.239.195
--webserver_port=0
--tserver_master_addrs=127.28.239.254:40129
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.195
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:30:17.671279 7568 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:30:17.672657 7568 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:30:17.687839 7581 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:30:17.687837 7583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:30:17.688485 7580 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:30:17.688997 7568 server_base.cc:1034] running on GCE node
I20260811 06:30:17.689673 7357 heartbeater.cc:503] Master 127.28.239.254:40129 was elected leader, sending a full tablet report...
I20260811 06:30:17.689785 7568 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:30:17.690963 7568 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:30:17.692240 7568 hybrid_clock.cc:648] HybridClock initialized: now 1786429817692156 us; error 66 us; skew 500 ppm
I20260811 06:30:17.692643 7568 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:30:17 dist-test-slave-gkw7 krb5kdc[6988](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429817, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.239.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:30:17.696146 7568 init.cc:377] Logged in from keytab as oryx/127.28.239.195@KRBTEST.COM (short username oryx)
I20260811 06:30:17.697803 7568 webserver.cc:460] Webserver started at http://127.28.239.195:37013/ using document root <none> and password file <none>
I20260811 06:30:17.698588 7568 fs_manager.cc:359] Metadata directory not provided
I20260811 06:30:17.698757 7568 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:30:17.699100 7568 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:30:17.702428 7568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-2/data/instance:
uuid: "7c0f7f24b8f0420a93c04bdf627b58e1"
format_stamp: "Formatted at 2026-08-11 06:30:17 on dist-test-slave-gkw7"
server_key: "7cdf79991a93a215d6c54600c142c9a2"
server_key_iv: "9bb7f1d7acb47984814262f2c3957857"
server_key_version: "encryptionkey@0"
I20260811 06:30:17.703341 7568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-2/wal/instance:
uuid: "7c0f7f24b8f0420a93c04bdf627b58e1"
format_stamp: "Formatted at 2026-08-11 06:30:17 on dist-test-slave-gkw7"
server_key: "7cdf79991a93a215d6c54600c142c9a2"
server_key_iv: "9bb7f1d7acb47984814262f2c3957857"
server_key_version: "encryptionkey@0"
I20260811 06:30:17.709303 7568 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 06:30:17.713440 7594 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:17.715005 7568 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:30:17.715224 7568 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-2/wal
uuid: "7c0f7f24b8f0420a93c04bdf627b58e1"
format_stamp: "Formatted at 2026-08-11 06:30:17 on dist-test-slave-gkw7"
server_key: "7cdf79991a93a215d6c54600c142c9a2"
server_key_iv: "9bb7f1d7acb47984814262f2c3957857"
server_key_version: "encryptionkey@0"
I20260811 06:30:17.715466 7568 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:30:17.735706 7568 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:30:17.740661 7568 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:30:17.740968 7568 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:30:17.742049 7568 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:30:17.743595 7568 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:30:17.743712 7568 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:17.743867 7568 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:30:17.743943 7568 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:17.780344 7568 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.195:44989
I20260811 06:30:17.780388 7764 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.195:44989 every 8 connection(s)
I20260811 06:30:17.781814 7568 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
I20260811 06:30:17.788738 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 7568
I20260811 06:30:17.789000 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-2/wal/instance
I20260811 06:30:17.789604 29631 external_mini_cluster.cc:1428] Setting key 56f553b330b9883ffcef6c2aeb68e388
Aug 11 06:30:17 dist-test-slave-gkw7 krb5kdc[6988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429817, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.239.195@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM
I20260811 06:30:17.804924 7767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:17.784745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.195:52719 (local address 127.28.239.254:40129)
0811 06:30:17.785238 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:17.785243 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:17.786306 (+ 1063us) server_negotiation.cc:407] Connection header received
0811 06:30:17.788769 (+ 2463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:17.788775 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:17.788892 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:17.789042 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:17.791242 (+ 2200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:17.792456 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:17.794259 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:17.794710 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:17.794821 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:17.798528 (+ 3707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:17.798533 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:17.798537 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:30:17.798572 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:17.801112 (+ 2540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:17.802065 (+ 953us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:17.802070 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:17.802072 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:17.802218 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:17.802709 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:17.802713 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:17.802715 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:17.802997 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:17.803127 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:17.804214 (+ 1087us) server_negotiation.cc:299] Negotiation successful
0811 06:30:17.804440 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":383,"thread_start_us":273,"threads_started":1}
I20260811 06:30:17.806237 7768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:17.785188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:40129 (local address 127.28.239.195:52719)
0811 06:30:17.786103 (+ 915us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:17.786155 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:17.788357 (+ 2202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:17.789369 (+ 1012us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:17.789382 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:17.790064 (+ 682us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:17.790853 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:17.790875 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:17.792625 (+ 1750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:17.792630 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:17.794117 (+ 1487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:17.794130 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:17.794914 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:17.794919 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:17.795116 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:17.795829 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:30:17.795851 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:17.798300 (+ 2449us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:30:17.801258 (+ 2958us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:30:17.801266 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:30:17.801282 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:30:17.801902 (+ 620us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:30:17.802365 (+ 463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:30:17.802369 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:30:17.802372 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:30:17.802596 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:30:17.803140 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:30:17.803147 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:30:17.803645 (+ 498us) client_negotiation.cc:769] Sending connection context
0811 06:30:17.803997 (+ 352us) client_negotiation.cc:240] Negotiation successful
0811 06:30:17.804597 (+ 600us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":654,"thread_start_us":443,"threads_started":1}
I20260811 06:30:17.807763 7765 heartbeater.cc:347] Connected to a master server at 127.28.239.254:40129
I20260811 06:30:17.808063 7765 heartbeater.cc:464] Registering TS with master...
I20260811 06:30:17.808857 7765 heartbeater.cc:511] Master 127.28.239.254:40129 requested a full tablet report, sending...
I20260811 06:30:17.810411 7048 ts_manager.cc:194] Registered new tserver with Master: 7c0f7f24b8f0420a93c04bdf627b58e1 (127.28.239.195:44989)
I20260811 06:30:17.811283 7048 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.28.239.195@KRBTEST.COM'} at 127.28.239.195:52719
I20260811 06:30:17.818871 29631 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.28.239.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.28.239.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.28.239.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/krb5kdc/oryx_127.28.239.253.keytab.
Entry for principal oryx/127.28.239.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/krb5kdc/oryx_127.28.239.253.keytab.
I20260811 06:30:17.879968 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.253:45951
--webserver_interface=127.28.239.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/krb5kdc/oryx_127.28.239.253.keytab
--principal=oryx/127.28.239.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:43555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.239.254:40129,127.28.239.253:45951
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 06:30:18.251165 7559 heartbeater.cc:503] Master 127.28.239.254:40129 was elected leader, sending a full tablet report...
W20260811 06:30:18.272215 7772 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:30:18.272667 7772 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:30:18.272810 7772 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:30:18.280879 7772 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:30:18.280993 7772 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:30:18.281085 7772 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:30:18.281188 7772 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:30:18.281260 7772 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:30:18.292586 7772 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:43555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.28.239.254:40129,127.28.239.253:45951
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/krb5kdc/oryx_127.28.239.253.keytab
--principal=oryx/127.28.239.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.253:45951
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.28.239.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
W20260811 06:30:18.293008 7772 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.28.239.254:40129,127.28.239.253:45951'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260811 06:30:18.294235 7772 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:30:18.295620 7772 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:30:18.311388 7780 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:30:18.311389 7781 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:30:18.312460 7783 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:30:18.313298 7772 server_base.cc:1034] running on GCE node
I20260811 06:30:18.314602 7772 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:30:18.316869 7772 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:30:18.318217 7772 hybrid_clock.cc:648] HybridClock initialized: now 1786429818318085 us; error 68 us; skew 500 ppm
I20260811 06:30:18.318894 7772 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:30:18 dist-test-slave-gkw7 krb5kdc[6988](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429818, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.239.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:30:18.323956 7772 init.cc:377] Logged in from keytab as oryx/127.28.239.253@KRBTEST.COM (short username oryx)
I20260811 06:30:18.325927 7772 webserver.cc:460] Webserver started at http://127.28.239.253:41801/ using document root <none> and password file <none>
I20260811 06:30:18.326803 7772 fs_manager.cc:359] Metadata directory not provided
I20260811 06:30:18.326908 7772 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:30:18.327325 7772 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:30:18.330932 7772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/master-1/data/instance:
uuid: "81d907d1c7ce487da25ae51b57e094e3"
format_stamp: "Formatted at 2026-08-11 06:30:18 on dist-test-slave-gkw7"
server_key: "845f93cd1870c08c1bd5fedc702a9f13"
server_key_iv: "79f5e7b7aee84e418f7c96703c8f999f"
server_key_version: "encryptionkey@0"
I20260811 06:30:18.331835 7772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/master-1/wal/instance:
uuid: "81d907d1c7ce487da25ae51b57e094e3"
format_stamp: "Formatted at 2026-08-11 06:30:18 on dist-test-slave-gkw7"
server_key: "845f93cd1870c08c1bd5fedc702a9f13"
server_key_iv: "79f5e7b7aee84e418f7c96703c8f999f"
server_key_version: "encryptionkey@0"
I20260811 06:30:18.338086 7772 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260811 06:30:18.342262 7792 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:18.343863 7772 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 06:30:18.344064 7772 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/master-1/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/master-1/wal
uuid: "81d907d1c7ce487da25ae51b57e094e3"
format_stamp: "Formatted at 2026-08-11 06:30:18 on dist-test-slave-gkw7"
server_key: "845f93cd1870c08c1bd5fedc702a9f13"
server_key_iv: "79f5e7b7aee84e418f7c96703c8f999f"
server_key_version: "encryptionkey@0"
I20260811 06:30:18.344303 7772 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:30:18.362535 7772 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:30:18.367035 7772 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:30:18.367322 7772 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:30:18.388789 7772 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.253:45951
I20260811 06:30:18.388810 7885 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.253:45951 every 8 connection(s)
I20260811 06:30:18.390344 7772 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/master-1/data/info.pb
I20260811 06:30:18.390933 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 7772
I20260811 06:30:18.396572 7886 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:30:18.410120 7886 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.239.254" port: 40129 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 06:30:18 dist-test-slave-gkw7 krb5kdc[6988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429815, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.28.239.253@KRBTEST.COM
I20260811 06:30:18.439687 7891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:18.394513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55872 (local address 127.28.239.253:45951)
0811 06:30:18.402246 (+ 7733us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:18.402269 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:18.408296 (+ 6027us) server_negotiation.cc:407] Connection header received
0811 06:30:18.411348 (+ 3052us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:18.411387 (+ 39us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:18.412069 (+ 682us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:18.412583 (+ 514us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:18.414641 (+ 2058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:18.417335 (+ 2694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:18.419984 (+ 2649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:18.421515 (+ 1531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:18.421801 (+ 286us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:18.429721 (+ 7920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:18.429740 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:18.429773 (+ 33us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:30:18.429853 (+ 80us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:18.434653 (+ 4800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:18.435561 (+ 908us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:18.435573 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:18.435586 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:18.435861 (+ 275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:18.436357 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:18.436361 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:18.436363 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:18.436902 (+ 539us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:18.437236 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:18.437772 (+ 536us) server_negotiation.cc:299] Negotiation successful
0811 06:30:18.438278 (+ 506us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7414,"thread_start_us":7136,"threads_started":1}
Aug 11 06:30:18 dist-test-slave-gkw7 krb5kdc[6988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429818, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.239.253@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM
I20260811 06:30:18.450122 7894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:18.414679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:40129 (local address 127.0.0.1:54034)
0811 06:30:18.431544 (+ 16865us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:18.431617 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:18.431874 (+ 257us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:18.432354 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:18.432362 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:18.433132 (+ 770us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:18.433428 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:18.433439 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:18.434525 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:18.434530 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:18.436133 (+ 1603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:18.436151 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:18.437610 (+ 1459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:18.437619 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:18.437888 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:18.438642 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:30:18.438682 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:18.441952 (+ 3270us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:30:18.446617 (+ 4665us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:30:18.446631 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:30:18.446637 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:30:18.447229 (+ 592us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:30:18.447750 (+ 521us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:30:18.447755 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:30:18.447758 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:30:18.447948 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:30:18.448983 (+ 1035us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:30:18.448989 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:30:18.449200 (+ 211us) client_negotiation.cc:769] Sending connection context
0811 06:30:18.449456 (+ 256us) client_negotiation.cc:240] Negotiation successful
0811 06:30:18.449792 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15739,"thread_start_us":15539,"threads_started":1}
I20260811 06:30:18.450560 7893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:18.414592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54034 (local address 127.28.239.254:40129)
0811 06:30:18.431384 (+ 16792us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:18.431390 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:18.431780 (+ 390us) server_negotiation.cc:407] Connection header received
0811 06:30:18.432021 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:18.432029 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:18.432147 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:18.432291 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:18.433633 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:18.434375 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:18.436314 (+ 1939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:18.437134 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:18.437281 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:18.442191 (+ 4910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:18.442202 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:18.442209 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:30:18.442269 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:18.446418 (+ 4149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:18.447393 (+ 975us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:18.447401 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:18.447407 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:18.447578 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:18.448095 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:18.448103 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:18.448109 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:18.448830 (+ 721us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:18.449010 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:18.449646 (+ 636us) server_negotiation.cc:299] Negotiation successful
0811 06:30:18.449962 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16633,"thread_start_us":16676,"threads_started":1}
I20260811 06:30:18.452337 7886 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.239.253" port: 45951 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 06:30:18 dist-test-slave-gkw7 krb5kdc[6988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429818, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.239.253@KRBTEST.COM for oryx/127.28.239.253@KRBTEST.COM
I20260811 06:30:18.469214 7894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:18.453274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.253:45951 (local address 127.0.0.1:55882)
0811 06:30:18.453497 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:18.453517 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:18.453714 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:18.454135 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:18.454141 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:18.454581 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:18.454750 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:18.454760 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:18.456018 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:18.456027 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:18.457095 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:18.457109 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:18.458061 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:18.458070 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:18.458268 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:18.458990 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:30:18.459011 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:18.461394 (+ 2383us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:30:18.466203 (+ 4809us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:30:18.466213 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:30:18.466218 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:30:18.466657 (+ 439us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:30:18.467113 (+ 456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:30:18.467120 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:30:18.467124 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:30:18.467291 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:30:18.467961 (+ 670us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:30:18.467967 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:30:18.468143 (+ 176us) client_negotiation.cc:769] Sending connection context
0811 06:30:18.468308 (+ 165us) client_negotiation.cc:240] Negotiation successful
0811 06:30:18.468676 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69}
I20260811 06:30:18.469290 7891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:18.453580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55882 (local address 127.28.239.253:45951)
0811 06:30:18.453763 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:18.453773 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:18.453811 (+ 38us) server_negotiation.cc:407] Connection header received
0811 06:30:18.453871 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:18.453875 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:18.453934 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:18.454058 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:18.454977 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:18.455815 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:18.457283 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:18.457846 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:18.458019 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:18.461736 (+ 3717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:18.461746 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:18.461754 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:30:18.461813 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:18.466037 (+ 4224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:18.466804 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:18.466813 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:18.466817 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:18.466962 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:18.467431 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:18.467438 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:18.467442 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:18.467819 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:18.468003 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:18.468410 (+ 407us) server_negotiation.cc:299] Negotiation successful
0811 06:30:18.468822 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":174,"server-negotiator.queue_time_us":62}
I20260811 06:30:18.475680 7886 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81d907d1c7ce487da25ae51b57e094e3: Bootstrap starting.
I20260811 06:30:18.479470 7886 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 81d907d1c7ce487da25ae51b57e094e3: Neither blocks nor log segments found. Creating new log.
I20260811 06:30:18.480767 7886 log.cc:824] T 00000000000000000000000000000000 P 81d907d1c7ce487da25ae51b57e094e3: Log is configured to *not* fsync() on all Append() calls
I20260811 06:30:18.484175 7886 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81d907d1c7ce487da25ae51b57e094e3: No bootstrap required, opened a new log
I20260811 06:30:18.492630 7886 raft_consensus.cc:348] T 00000000000000000000000000000000 P 81d907d1c7ce487da25ae51b57e094e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1478483ef29416fb130a1d621aafbaa" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 40129 } } peers { permanent_uuid: "81d907d1c7ce487da25ae51b57e094e3" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 45951 } }
I20260811 06:30:18.492978 7886 raft_consensus.cc:374] T 00000000000000000000000000000000 P 81d907d1c7ce487da25ae51b57e094e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:30:18.493089 7886 raft_consensus.cc:729] T 00000000000000000000000000000000 P 81d907d1c7ce487da25ae51b57e094e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81d907d1c7ce487da25ae51b57e094e3, State: Initialized, Role: FOLLOWER
I20260811 06:30:18.493816 7886 consensus_queue.cc:260] T 00000000000000000000000000000000 P 81d907d1c7ce487da25ae51b57e094e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1478483ef29416fb130a1d621aafbaa" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 40129 } } peers { permanent_uuid: "81d907d1c7ce487da25ae51b57e094e3" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 45951 } }
I20260811 06:30:18.494845 7896 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81d907d1c7ce487da25ae51b57e094e3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1478483ef29416fb130a1d621aafbaa" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 40129 } } peers { permanent_uuid: "81d907d1c7ce487da25ae51b57e094e3" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 45951 } } }
I20260811 06:30:18.495229 7896 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81d907d1c7ce487da25ae51b57e094e3 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:30:18.495890 7886 sys_catalog.cc:564] T 00000000000000000000000000000000 P 81d907d1c7ce487da25ae51b57e094e3 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 06:30:18.505339 7912 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 81d907d1c7ce487da25ae51b57e094e3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 06:30:18.505486 7912 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 06:30:18.509450 7772 master_runner.cc:428] Detected that this master 81d907d1c7ce487da25ae51b57e094e3 is joining an existing cluster
I20260811 06:30:18.509603 7772 master_runner.cc:432] Initiating AddMaster RPC to add 127.28.239.253:45951
I20260811 06:30:18.530043 7918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:18.513779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.253:45951 (local address 127.0.0.1:55894)
0811 06:30:18.514552 (+ 773us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:18.514592 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:18.514804 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:18.515226 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:18.515233 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:18.516039 (+ 806us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:18.516489 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:18.516502 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:18.518221 (+ 1719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:18.518232 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:18.519860 (+ 1628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:18.519878 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:18.520757 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:18.520763 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:18.520949 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:18.521753 (+ 804us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:30:18.521793 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:18.523333 (+ 1540us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:30:18.526788 (+ 3455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:30:18.526796 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:30:18.526801 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:30:18.527228 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:30:18.527695 (+ 467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:30:18.527702 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:30:18.527705 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:30:18.527847 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:30:18.528796 (+ 949us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:30:18.528800 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:30:18.528982 (+ 182us) client_negotiation.cc:769] Sending connection context
0811 06:30:18.529178 (+ 196us) client_negotiation.cc:240] Negotiation successful
0811 06:30:18.529511 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":594,"mutex_wait_us":4,"spinlock_wait_cycles":23296,"thread_start_us":426,"threads_started":1}
I20260811 06:30:18.530138 7891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:18.513699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55894 (local address 127.28.239.253:45951)
0811 06:30:18.513895 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:18.513901 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:18.514684 (+ 783us) server_negotiation.cc:407] Connection header received
0811 06:30:18.514964 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:18.514970 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:18.515047 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:18.515176 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:18.516671 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:18.517962 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:18.520050 (+ 2088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:18.520572 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:18.520714 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:18.523517 (+ 2803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:18.523522 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:18.523525 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:30:18.523588 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:18.526652 (+ 3064us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:18.527351 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:18.527358 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:18.527363 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:18.527587 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:18.527993 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:18.527997 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:18.528001 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:18.528362 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:18.528533 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:18.529284 (+ 751us) server_negotiation.cc:299] Negotiation successful
0811 06:30:18.529722 (+ 438us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":216,"server-negotiator.queue_time_us":67}
I20260811 06:30:18.531008 7893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:18.513522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54048 (local address 127.28.239.254:40129)
0811 06:30:18.513757 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:18.513764 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:18.515047 (+ 1283us) server_negotiation.cc:407] Connection header received
0811 06:30:18.515295 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:18.515302 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:18.515404 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:18.515481 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:18.516958 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:18.517738 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:18.520024 (+ 2286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:18.520780 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:18.520911 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:18.523504 (+ 2593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:18.523511 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:18.523517 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:30:18.523557 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:18.527397 (+ 3840us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:18.528209 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:18.528214 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:18.528216 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:18.528358 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:18.528771 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:18.528774 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:18.528776 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:18.529063 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:18.529229 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:18.530281 (+ 1052us) server_negotiation.cc:299] Negotiation successful
0811 06:30:18.530704 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260811 06:30:18.531589 7919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:18.514197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:40129 (local address 127.0.0.1:54048)
0811 06:30:18.514943 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:18.514962 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:18.515115 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:18.515732 (+ 617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:18.515741 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:18.516441 (+ 700us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:18.516813 (+ 372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:18.516821 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:18.517888 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:18.517895 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:18.519860 (+ 1965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:18.519875 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:18.521078 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:18.521086 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:18.521306 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:18.521942 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:30:18.521964 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:18.523298 (+ 1334us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:30:18.527556 (+ 4258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:30:18.527562 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:30:18.527566 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:30:18.528081 (+ 515us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:30:18.528486 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:30:18.528491 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:30:18.528496 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:30:18.528661 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:30:18.529392 (+ 731us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:30:18.529395 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:30:18.529605 (+ 210us) client_negotiation.cc:769] Sending connection context
0811 06:30:18.529747 (+ 142us) client_negotiation.cc:240] Negotiation successful
0811 06:30:18.530051 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":607,"mutex_wait_us":26,"spinlock_wait_cycles":30976,"thread_start_us":423,"threads_started":1}
Aug 11 06:30:18 dist-test-slave-gkw7 krb5kdc[6988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429815, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.239.254@KRBTEST.COM for oryx/127.28.239.253@KRBTEST.COM
I20260811 06:30:18.557987 7891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:18.538587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55898 (local address 127.28.239.253:45951)
0811 06:30:18.538820 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:18.538829 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:18.539635 (+ 806us) server_negotiation.cc:407] Connection header received
0811 06:30:18.539941 (+ 306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:18.539946 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:18.540093 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:18.540207 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:18.541661 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:18.542378 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:18.544328 (+ 1950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:18.544790 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:18.544940 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:18.549406 (+ 4466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:18.549416 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:18.549428 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:30:18.549480 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:18.553893 (+ 4413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:18.555034 (+ 1141us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:18.555041 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:18.555044 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:18.555226 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:18.555813 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:18.555821 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:18.555827 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:18.556338 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:18.556505 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:18.557218 (+ 713us) server_negotiation.cc:299] Negotiation successful
0811 06:30:18.557503 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260811 06:30:18.557999 7921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:18.538692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.253:45951 (local address 127.0.0.1:55898)
0811 06:30:18.539457 (+ 765us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:18.539525 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:18.539707 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:18.540283 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:18.540291 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:18.541164 (+ 873us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:18.541461 (+ 297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:18.541474 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:18.542519 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:18.542525 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:18.544158 (+ 1633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:18.544174 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:18.544958 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:18.544965 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:18.545177 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:18.545944 (+ 767us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:30:18.545967 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:18.549160 (+ 3193us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:30:18.554085 (+ 4925us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:30:18.554102 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:30:18.554111 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:30:18.554828 (+ 717us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:30:18.555376 (+ 548us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:30:18.555383 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:30:18.555389 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:30:18.555628 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:30:18.556547 (+ 919us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:30:18.556558 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:30:18.556837 (+ 279us) client_negotiation.cc:769] Sending connection context
0811 06:30:18.557087 (+ 250us) client_negotiation.cc:240] Negotiation successful
0811 06:30:18.557488 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":297,"threads_started":1}
I20260811 06:30:18.560649 7048 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.28.239.253:45951: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "81d907d1c7ce487da25ae51b57e094e3"
member_type: NON_VOTER
last_known_addr {
host: "127.28.239.253"
port: 45951
}
attrs {
promote: true
}
}
dest_uuid: "c1478483ef29416fb130a1d621aafbaa"
cas_config_opid_index: -1
I20260811 06:30:18.562106 7048 consensus_queue.cc:237] T 00000000000000000000000000000000 P c1478483ef29416fb130a1d621aafbaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c1478483ef29416fb130a1d621aafbaa" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 40129 } } peers { permanent_uuid: "81d907d1c7ce487da25ae51b57e094e3" member_type: NON_VOTER last_known_addr { host: "127.28.239.253" port: 45951 } attrs { promote: true } }
I20260811 06:30:18.573709 7838 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 81d907d1c7ce487da25ae51b57e094e3 [term 0 FOLLOWER]: Advancing to term 1
WARNING: no policy specified for oryx/127.28.239.252@KRBTEST.COM; defaulting to no policy
I20260811 06:30:18.575978 7924 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c1478483ef29416fb130a1d621aafbaa [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 81d907d1c7ce487da25ae51b57e094e3 (127.28.239.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c1478483ef29416fb130a1d621aafbaa" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 40129 } } peers { permanent_uuid: "81d907d1c7ce487da25ae51b57e094e3" member_type: NON_VOTER last_known_addr { host: "127.28.239.253" port: 45951 } attrs { promote: true } } }
I20260811 06:30:18.578845 7838 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 81d907d1c7ce487da25ae51b57e094e3 [term 1 FOLLOWER]: Refusing update from remote peer c1478483ef29416fb130a1d621aafbaa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.28.239.252@KRBTEST.COM" created.
I20260811 06:30:18.580086 7926 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c1478483ef29416fb130a1d621aafbaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "81d907d1c7ce487da25ae51b57e094e3" member_type: NON_VOTER last_known_addr { host: "127.28.239.253" port: 45951 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:30:18.596045 7924 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.28.239.253:45951
I20260811 06:30:18.598255 7926 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1478483ef29416fb130a1d621aafbaa [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c1478483ef29416fb130a1d621aafbaa" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c1478483ef29416fb130a1d621aafbaa" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 40129 } } peers { permanent_uuid: "81d907d1c7ce487da25ae51b57e094e3" member_type: NON_VOTER last_known_addr { host: "127.28.239.253" port: 45951 } attrs { promote: true } } }
I20260811 06:30:18.598611 7772 master.cc:537] Master@127.28.239.253:45951 shutting down...
I20260811 06:30:18.598605 7926 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1478483ef29416fb130a1d621aafbaa [sys.catalog]: This master's current role is: LEADER
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.28.239.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/krb5kdc/oryx_127.28.239.252.keytab.
Entry for principal oryx/127.28.239.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/krb5kdc/oryx_127.28.239.252.keytab.
I20260811 06:30:18.630829 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.252:43777
--webserver_interface=127.28.239.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/krb5kdc/oryx_127.28.239.252.keytab
--principal=oryx/127.28.239.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:43555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.239.254:40129,127.28.239.253:45951,127.28.239.252:43777
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:30:18.645924 7802 tablet.cc:2369] T 00000000000000000000000000000000 P 81d907d1c7ce487da25ae51b57e094e3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:30:18.652583 7838 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 81d907d1c7ce487da25ae51b57e094e3 [term 1 LEARNER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 81d907d1c7ce487da25ae51b57e094e3 (127.28.239.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c1478483ef29416fb130a1d621aafbaa" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 40129 } } peers { permanent_uuid: "81d907d1c7ce487da25ae51b57e094e3" member_type: NON_VOTER last_known_addr { host: "127.28.239.253" port: 45951 } attrs { promote: true } } }
I20260811 06:30:18.658995 7896 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81d907d1c7ce487da25ae51b57e094e3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c1478483ef29416fb130a1d621aafbaa. Latest consensus state: current_term: 1 leader_uuid: "c1478483ef29416fb130a1d621aafbaa" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c1478483ef29416fb130a1d621aafbaa" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 40129 } } peers { permanent_uuid: "81d907d1c7ce487da25ae51b57e094e3" member_type: NON_VOTER last_known_addr { host: "127.28.239.253" port: 45951 } attrs { promote: true } } }
I20260811 06:30:18.659353 7896 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81d907d1c7ce487da25ae51b57e094e3 [sys.catalog]: This master's current role is: LEARNER
I20260811 06:30:18.658973 7937 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81d907d1c7ce487da25ae51b57e094e3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c1478483ef29416fb130a1d621aafbaa" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c1478483ef29416fb130a1d621aafbaa" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 40129 } } peers { permanent_uuid: "81d907d1c7ce487da25ae51b57e094e3" member_type: NON_VOTER last_known_addr { host: "127.28.239.253" port: 45951 } attrs { promote: true } } }
I20260811 06:30:18.659914 7937 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81d907d1c7ce487da25ae51b57e094e3 [sys.catalog]: This master's current role is: LEARNER
I20260811 06:30:18.658759 7940 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81d907d1c7ce487da25ae51b57e094e3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c1478483ef29416fb130a1d621aafbaa" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c1478483ef29416fb130a1d621aafbaa" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 40129 } } peers { permanent_uuid: "81d907d1c7ce487da25ae51b57e094e3" member_type: NON_VOTER last_known_addr { host: "127.28.239.253" port: 45951 } attrs { promote: true } } }
I20260811 06:30:18.660629 7940 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81d907d1c7ce487da25ae51b57e094e3 [sys.catalog]: This master's current role is: LEARNER
I20260811 06:30:18.662786 7772 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 81d907d1c7ce487da25ae51b57e094e3 [term 1 LEARNER]: Raft consensus shutting down.
I20260811 06:30:18.663309 7772 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 81d907d1c7ce487da25ae51b57e094e3 [term 1 LEARNER]: Raft consensus is shut down!
I20260811 06:30:18.663473 7772 tablet_replica.cc:330] T 00000000000000000000000000000000 P 81d907d1c7ce487da25ae51b57e094e3: stopping tablet replica
I20260811 06:30:18.671895 7772 master.cc:552] Master@127.28.239.253:45951 shutdown complete.
I20260811 06:30:18.672026 7772 master_runner.cc:305] Clearing existing system tablet
I20260811 06:30:18.672497 7772 fs_manager.cc:359] Metadata directory not provided
I20260811 06:30:18.672595 7772 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:30:18.675724 7772 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.001s
I20260811 06:30:18.681383 7942 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:18.681831 7772 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:30:18.682199 7772 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.005s
I20260811 06:30:18.682319 7772 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/master-1/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/master-1/wal
uuid: "81d907d1c7ce487da25ae51b57e094e3"
format_stamp: "Formatted at 2026-08-11 06:30:18 on dist-test-slave-gkw7"
server_key: "845f93cd1870c08c1bd5fedc702a9f13"
server_key_iv: "79f5e7b7aee84e418f7c96703c8f999f"
server_key_version: "encryptionkey@0"
I20260811 06:30:18.687470 7772 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 81d907d1c7ce487da25ae51b57e094e3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:30:18.695678 7772 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 81d907d1c7ce487da25ae51b57e094e3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 06:30:18.696051 7772 log.cc:1196] T 00000000000000000000000000000000 P 81d907d1c7ce487da25ae51b57e094e3: Deleting WAL directory at /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260811 06:30:18.696771 7772 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 81d907d1c7ce487da25ae51b57e094e3: Deleting consensus metadata
I20260811 06:30:18.697129 7772 master_runner.cc:315] Copying system tablet from 127.28.239.254:40129
I20260811 06:30:18.700172 7772 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 81d907d1c7ce487da25ae51b57e094e3: tablet copy: Beginning tablet copy session from remote peer at address 127.28.239.254:40129
I20260811 06:30:18.717965 7893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:18.701236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54062 (local address 127.28.239.254:40129)
0811 06:30:18.701442 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:18.701451 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:18.702184 (+ 733us) server_negotiation.cc:407] Connection header received
0811 06:30:18.702533 (+ 349us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:18.702540 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:18.702660 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:18.702752 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:18.704325 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:18.705647 (+ 1322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:18.707559 (+ 1912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:18.708241 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:18.708380 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:18.710883 (+ 2503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:18.710892 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:18.710896 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:30:18.710938 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:18.714459 (+ 3521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:18.715329 (+ 870us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:18.715336 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:18.715341 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:18.715518 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:18.715986 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:18.715990 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:18.715993 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:18.716368 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:18.716495 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:18.717120 (+ 625us) server_negotiation.cc:299] Negotiation successful
0811 06:30:18.717418 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 06:30:18.719435 7953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:18.701167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:40129 (local address 127.0.0.1:54062)
0811 06:30:18.702014 (+ 847us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:18.702051 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:18.702333 (+ 282us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:18.703042 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:18.703048 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:18.703730 (+ 682us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:18.704086 (+ 356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:18.704107 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:18.705830 (+ 1723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:18.705835 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:18.707373 (+ 1538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:18.707390 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:18.708447 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:18.708454 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:18.708653 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:18.709421 (+ 768us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:30:18.709447 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:18.710669 (+ 1222us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:30:18.714641 (+ 3972us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:30:18.714646 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:30:18.714650 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:30:18.715177 (+ 527us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:30:18.715676 (+ 499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:30:18.715680 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:30:18.715684 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:30:18.715847 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:30:18.716607 (+ 760us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:30:18.716612 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:30:18.716793 (+ 181us) client_negotiation.cc:769] Sending connection context
0811 06:30:18.718540 (+ 1747us) client_negotiation.cc:240] Negotiation successful
0811 06:30:18.718894 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":647,"thread_start_us":473,"threads_started":1}
I20260811 06:30:18.720629 7080 tablet_copy_service.cc:140] P c1478483ef29416fb130a1d621aafbaa: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 81d907d1c7ce487da25ae51b57e094e3 ({username='oryx', principal='oryx/127.28.239.253@KRBTEST.COM'} at 127.0.0.1:54062)
I20260811 06:30:18.720955 7080 tablet_copy_service.cc:161] P c1478483ef29416fb130a1d621aafbaa: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 81d907d1c7ce487da25ae51b57e094e3 at {username='oryx', principal='oryx/127.28.239.253@KRBTEST.COM'} at 127.0.0.1:54062: session id = 81d907d1c7ce487da25ae51b57e094e3-00000000000000000000000000000000
I20260811 06:30:18.725658 7080 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P c1478483ef29416fb130a1d621aafbaa: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 06:30:18.729262 7772 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 06:30:18.736487 7772 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 81d907d1c7ce487da25ae51b57e094e3: tablet copy: Starting download of 0 data blocks...
I20260811 06:30:18.736842 7772 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 81d907d1c7ce487da25ae51b57e094e3: tablet copy: Starting download of 1 WAL segments...
I20260811 06:30:18.740523 7772 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 81d907d1c7ce487da25ae51b57e094e3: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 06:30:18.745802 7080 tablet_copy_service.cc:342] P c1478483ef29416fb130a1d621aafbaa: Request end of tablet copy session 81d907d1c7ce487da25ae51b57e094e3-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.28.239.253@KRBTEST.COM'} at 127.0.0.1:54062
I20260811 06:30:18.746045 7080 tablet_copy_service.cc:433] P c1478483ef29416fb130a1d621aafbaa: ending tablet copy session 81d907d1c7ce487da25ae51b57e094e3-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 81d907d1c7ce487da25ae51b57e094e3
I20260811 06:30:18.758774 7772 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:30:18.761804 7954 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:30:18.767155 7956 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:30:18.775287 7772 server_base.cc:1034] running on GCE node
W20260811 06:30:18.775857 7958 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:30:18.776448 7772 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:30:18.777130 7772 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:30:18.778412 7772 hybrid_clock.cc:648] HybridClock initialized: now 1786429818778341 us; error 41 us; skew 500 ppm
I20260811 06:30:18.778672 7772 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:30:18 dist-test-slave-gkw7 krb5kdc[6988](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429818, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.239.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:30:18.781024 7772 init.cc:377] Logged in from keytab as oryx/127.28.239.253@KRBTEST.COM (short username oryx)
I20260811 06:30:18.788870 7772 webserver.cc:460] Webserver started at http://127.28.239.253:35229/ using document root <none> and password file <none>
I20260811 06:30:18.789390 7772 fs_manager.cc:359] Metadata directory not provided
I20260811 06:30:18.789491 7772 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:30:18.793105 7772 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260811 06:30:18.795413 7966 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:18.795889 7772 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 06:30:18.796036 7772 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/master-1/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/master-1/wal
uuid: "81d907d1c7ce487da25ae51b57e094e3"
format_stamp: "Formatted at 2026-08-11 06:30:18 on dist-test-slave-gkw7"
server_key: "845f93cd1870c08c1bd5fedc702a9f13"
server_key_iv: "79f5e7b7aee84e418f7c96703c8f999f"
server_key_version: "encryptionkey@0"
I20260811 06:30:18.796212 7772 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:30:18.814059 7765 heartbeater.cc:503] Master 127.28.239.254:40129 was elected leader, sending a full tablet report...
I20260811 06:30:18.818787 7772 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:30:18.823469 7772 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:30:18.870600 7772 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.253:45951
I20260811 06:30:18.870688 8048 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.253:45951 every 8 connection(s)
I20260811 06:30:18.871891 7772 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/master-1/data/info.pb
I20260811 06:30:18.878021 8051 sys_catalog.cc:263] Verifying existing consensus state
I20260811 06:30:18.882125 8051 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81d907d1c7ce487da25ae51b57e094e3: Bootstrap starting.
I20260811 06:30:18.899763 8051 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81d907d1c7ce487da25ae51b57e094e3: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 06:30:18.900513 8051 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81d907d1c7ce487da25ae51b57e094e3: Bootstrap complete.
I20260811 06:30:18.902009 8051 raft_consensus.cc:348] T 00000000000000000000000000000000 P 81d907d1c7ce487da25ae51b57e094e3 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c1478483ef29416fb130a1d621aafbaa" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 40129 } } peers { permanent_uuid: "81d907d1c7ce487da25ae51b57e094e3" member_type: NON_VOTER last_known_addr { host: "127.28.239.253" port: 45951 } attrs { promote: true } }
I20260811 06:30:18.902345 8051 raft_consensus.cc:729] T 00000000000000000000000000000000 P 81d907d1c7ce487da25ae51b57e094e3 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 81d907d1c7ce487da25ae51b57e094e3, State: Initialized, Role: LEARNER
I20260811 06:30:18.902670 8051 consensus_queue.cc:260] T 00000000000000000000000000000000 P 81d907d1c7ce487da25ae51b57e094e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c1478483ef29416fb130a1d621aafbaa" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 40129 } } peers { permanent_uuid: "81d907d1c7ce487da25ae51b57e094e3" member_type: NON_VOTER last_known_addr { host: "127.28.239.253" port: 45951 } attrs { promote: true } }
I20260811 06:30:18.903909 8058 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81d907d1c7ce487da25ae51b57e094e3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c1478483ef29416fb130a1d621aafbaa" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 40129 } } peers { permanent_uuid: "81d907d1c7ce487da25ae51b57e094e3" member_type: NON_VOTER last_known_addr { host: "127.28.239.253" port: 45951 } attrs { promote: true } } }
I20260811 06:30:18.904331 8058 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81d907d1c7ce487da25ae51b57e094e3 [sys.catalog]: This master's current role is: LEARNER
I20260811 06:30:18.904392 8051 sys_catalog.cc:564] T 00000000000000000000000000000000 P 81d907d1c7ce487da25ae51b57e094e3 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:30:18.916597 8073 catalog_manager.cc:1259] Loaded cluster ID: e6f352fda9a24973ae7bc5beafb7d908
I20260811 06:30:18.916723 8073 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 81d907d1c7ce487da25ae51b57e094e3: loading cluster ID for follower catalog manager: success
I20260811 06:30:18.920567 8073 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 81d907d1c7ce487da25ae51b57e094e3: acquiring CA information for follower catalog manager: success
I20260811 06:30:18.923434 8073 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 81d907d1c7ce487da25ae51b57e094e3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260811 06:30:19.061094 7936 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:30:19.061461 7936 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:30:19.061654 7936 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:30:19.069057 7936 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:30:19.069227 7936 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:30:19.069355 7936 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:30:19.069456 7936 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:30:19.069561 7936 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:30:19.079946 7936 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:43555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.28.239.254:40129,127.28.239.253:45951,127.28.239.252:43777
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/krb5kdc/oryx_127.28.239.252.keytab
--principal=oryx/127.28.239.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.252:43777
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.28.239.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:30:19.081384 7936 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:30:19.082813 7936 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:30:19.095140 8081 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:30:19.095168 8083 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:30:19.095388 8086 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:30:19.096453 7936 server_base.cc:1034] running on GCE node
I20260811 06:30:19.097307 7936 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:30:19.098881 7936 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:30:19.100113 7936 hybrid_clock.cc:648] HybridClock initialized: now 1786429819100050 us; error 64 us; skew 500 ppm
I20260811 06:30:19.100482 7936 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:30:19 dist-test-slave-gkw7 krb5kdc[6988](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429819, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.239.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:30:19.104043 7936 init.cc:377] Logged in from keytab as oryx/127.28.239.252@KRBTEST.COM (short username oryx)
I20260811 06:30:19.105844 7936 webserver.cc:460] Webserver started at http://127.28.239.252:43503/ using document root <none> and password file <none>
I20260811 06:30:19.106598 7936 fs_manager.cc:359] Metadata directory not provided
I20260811 06:30:19.106737 7936 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:30:19.107059 7936 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:30:19.110323 7936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/master-2/data/instance:
uuid: "3dbd88a041b4459980dab85a3d3bfaf7"
format_stamp: "Formatted at 2026-08-11 06:30:19 on dist-test-slave-gkw7"
server_key: "89d1c39b4025c7532c0a8c10a8dc02f2"
server_key_iv: "6d2c865ef92a5233b9d348fbf9979e26"
server_key_version: "encryptionkey@0"
I20260811 06:30:19.111153 7936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/master-2/wal/instance:
uuid: "3dbd88a041b4459980dab85a3d3bfaf7"
format_stamp: "Formatted at 2026-08-11 06:30:19 on dist-test-slave-gkw7"
server_key: "89d1c39b4025c7532c0a8c10a8dc02f2"
server_key_iv: "6d2c865ef92a5233b9d348fbf9979e26"
server_key_version: "encryptionkey@0"
I20260811 06:30:19.117004 7936 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 06:30:19.121055 8096 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:19.122824 7936 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:30:19.123034 7936 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/master-2/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/master-2/wal
uuid: "3dbd88a041b4459980dab85a3d3bfaf7"
format_stamp: "Formatted at 2026-08-11 06:30:19 on dist-test-slave-gkw7"
server_key: "89d1c39b4025c7532c0a8c10a8dc02f2"
server_key_iv: "6d2c865ef92a5233b9d348fbf9979e26"
server_key_version: "encryptionkey@0"
I20260811 06:30:19.123270 7936 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:30:19.142112 7936 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:30:19.146442 7936 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:30:19.146746 7936 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:30:19.166577 7936 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.252:43777
I20260811 06:30:19.166702 8165 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.252:43777 every 8 connection(s)
I20260811 06:30:19.168002 7936 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/master-2/data/info.pb
I20260811 06:30:19.170966 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 7936
I20260811 06:30:19.173374 8166 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:30:19.182528 8166 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.239.254" port: 40129 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 06:30:19 dist-test-slave-gkw7 krb5kdc[6988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429815, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.28.239.252@KRBTEST.COM
Aug 11 06:30:19 dist-test-slave-gkw7 krb5kdc[6988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429819, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.239.252@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM
I20260811 06:30:19.197693 8172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:19.173388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45166 (local address 127.28.239.252:43777)
0811 06:30:19.174367 (+ 979us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:19.174379 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:19.174425 (+ 46us) server_negotiation.cc:407] Connection header received
0811 06:30:19.176175 (+ 1750us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:19.176198 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:19.176633 (+ 435us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:19.177006 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:19.178934 (+ 1928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:19.180422 (+ 1488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:19.182292 (+ 1870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:19.183034 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:19.183186 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:19.188491 (+ 5305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:19.188498 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:19.188509 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:30:19.188544 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:19.192352 (+ 3808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:19.193250 (+ 898us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:19.193259 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:19.193275 (+ 16us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:19.193471 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:19.193979 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:19.193984 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:19.193987 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:19.194578 (+ 591us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:19.194818 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:19.195549 (+ 731us) server_negotiation.cc:299] Negotiation successful
0811 06:30:19.196000 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":759,"thread_start_us":432,"threads_started":1}
I20260811 06:30:19.202841 8177 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1478483ef29416fb130a1d621aafbaa [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c1478483ef29416fb130a1d621aafbaa" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c1478483ef29416fb130a1d621aafbaa" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 40129 } } peers { permanent_uuid: "81d907d1c7ce487da25ae51b57e094e3" member_type: NON_VOTER last_known_addr { host: "127.28.239.253" port: 45951 } attrs { promote: true } } }
I20260811 06:30:19.203225 8177 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1478483ef29416fb130a1d621aafbaa [sys.catalog]: This master's current role is: LEADER
I20260811 06:30:19.206684 7893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:19.185806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54066 (local address 127.28.239.254:40129)
0811 06:30:19.186121 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:19.186129 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:19.186681 (+ 552us) server_negotiation.cc:407] Connection header received
0811 06:30:19.186903 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:19.186909 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:19.186989 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:19.187119 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:19.188251 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:19.189051 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:19.191346 (+ 2295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:19.191764 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:19.191864 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:19.195689 (+ 3825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:19.195695 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:19.195698 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:30:19.195767 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:19.202487 (+ 6720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:19.203795 (+ 1308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:19.203803 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:19.203810 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:19.204020 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:19.204557 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:19.204565 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:19.204571 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:19.205353 (+ 782us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:19.205481 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:19.206250 (+ 769us) server_negotiation.cc:299] Negotiation successful
0811 06:30:19.206432 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":68,"server-negotiator.queue_time_us":88}
I20260811 06:30:19.206965 8173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:19.185698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:40129 (local address 127.0.0.1:54066)
0811 06:30:19.186501 (+ 803us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:19.186552 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:19.186766 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:19.187166 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:19.187171 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:19.187842 (+ 671us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:19.188087 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:19.188096 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:19.189202 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:19.189206 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:19.191164 (+ 1958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:19.191179 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:19.191974 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:19.191983 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:19.192205 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:19.192869 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:30:19.192886 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:19.195482 (+ 2596us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:30:19.202695 (+ 7213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:30:19.202713 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:30:19.202725 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:30:19.203548 (+ 823us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:30:19.204185 (+ 637us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:30:19.204191 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:30:19.204195 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:30:19.204393 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:30:19.205507 (+ 1114us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:30:19.205517 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:30:19.205836 (+ 319us) client_negotiation.cc:769] Sending connection context
0811 06:30:19.206114 (+ 278us) client_negotiation.cc:240] Negotiation successful
0811 06:30:19.206522 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":537,"thread_start_us":378,"threads_started":1}
I20260811 06:30:19.208510 8166 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.239.253" port: 45951 } has no permanent_uuid. Determining permanent_uuid...
I20260811 06:30:19.212026 8184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:19.198092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55904 (local address 127.28.239.253:45951)
0811 06:30:19.204269 (+ 6177us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:19.204279 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:19.204302 (+ 23us) server_negotiation.cc:407] Connection header received
0811 06:30:19.204385 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:19.204390 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:19.204524 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:19.204743 (+ 219us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:30:19.205628 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:19.207611 (+ 1983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:19.209888 (+ 2277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:19.210955 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:19.211184 (+ 229us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:19.211406 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:19.211540 (+ 134us) server_negotiation.cc:299] Negotiation successful
0811 06:30:19.211681 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6029,"thread_start_us":5822,"threads_started":1}
I20260811 06:30:19.212069 8182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:19.197890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.253:45951 (local address 127.0.0.1:55904)
0811 06:30:19.202758 (+ 4868us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:19.202780 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:19.202932 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:19.204735 (+ 1803us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:19.204740 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:19.204771 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:30:19.205360 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:19.205376 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:19.207782 (+ 2406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:19.207789 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:19.209646 (+ 1857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:19.209665 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:19.211148 (+ 1483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:19.211155 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:19.211342 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:19.211407 (+ 65us) client_negotiation.cc:769] Sending connection context
0811 06:30:19.211501 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 06:30:19.211653 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4737,"mutex_wait_us":68,"thread_start_us":4570,"threads_started":1}
I20260811 06:30:19.213040 8058 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81d907d1c7ce487da25ae51b57e094e3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c1478483ef29416fb130a1d621aafbaa. Latest consensus state: current_term: 1 leader_uuid: "c1478483ef29416fb130a1d621aafbaa" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c1478483ef29416fb130a1d621aafbaa" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 40129 } } peers { permanent_uuid: "81d907d1c7ce487da25ae51b57e094e3" member_type: NON_VOTER last_known_addr { host: "127.28.239.253" port: 45951 } attrs { promote: true } } }
I20260811 06:30:19.213361 8058 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81d907d1c7ce487da25ae51b57e094e3 [sys.catalog]: This master's current role is: LEARNER
I20260811 06:30:19.214270 8178 raft_consensus.cc:1050] T 00000000000000000000000000000000 P c1478483ef29416fb130a1d621aafbaa: attempting to promote NON_VOTER 81d907d1c7ce487da25ae51b57e094e3 to VOTER
I20260811 06:30:19.215082 8178 consensus_queue.cc:237] T 00000000000000000000000000000000 P c1478483ef29416fb130a1d621aafbaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c1478483ef29416fb130a1d621aafbaa" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 40129 } } peers { permanent_uuid: "81d907d1c7ce487da25ae51b57e094e3" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 45951 } attrs { promote: false } }
I20260811 06:30:19.217317 8012 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 81d907d1c7ce487da25ae51b57e094e3 [term 1 LEARNER]: Refusing update from remote peer c1478483ef29416fb130a1d621aafbaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
Aug 11 06:30:19 dist-test-slave-gkw7 krb5kdc[6988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429819, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.239.252@KRBTEST.COM for oryx/127.28.239.253@KRBTEST.COM
I20260811 06:30:19.217945 8178 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c1478483ef29416fb130a1d621aafbaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "81d907d1c7ce487da25ae51b57e094e3" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 45951 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260811 06:30:19.221381 8180 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c1478483ef29416fb130a1d621aafbaa [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 81d907d1c7ce487da25ae51b57e094e3 (127.28.239.253) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c1478483ef29416fb130a1d621aafbaa" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 40129 } } peers { permanent_uuid: "81d907d1c7ce487da25ae51b57e094e3" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 45951 } attrs { promote: false } } }
I20260811 06:30:19.222157 8012 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 81d907d1c7ce487da25ae51b57e094e3 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 81d907d1c7ce487da25ae51b57e094e3 (127.28.239.253) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c1478483ef29416fb130a1d621aafbaa" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 40129 } } peers { permanent_uuid: "81d907d1c7ce487da25ae51b57e094e3" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 45951 } attrs { promote: false } } }
I20260811 06:30:19.225816 8173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:19.209505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.253:45951 (local address 127.0.0.1:55918)
0811 06:30:19.209751 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:19.209772 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:19.209888 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:19.210529 (+ 641us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:19.210536 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:19.210969 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:19.211099 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:19.211107 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:19.212347 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:19.212352 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:19.213845 (+ 1493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:19.213863 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:19.214950 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:19.214959 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:19.215174 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:19.215879 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:30:19.215897 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:19.218281 (+ 2384us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:30:19.222743 (+ 4462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:30:19.222749 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:30:19.222755 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:30:19.223403 (+ 648us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:30:19.223853 (+ 450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:30:19.223857 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:30:19.223861 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:30:19.224053 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:30:19.224736 (+ 683us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:30:19.224741 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:30:19.224904 (+ 163us) client_negotiation.cc:769] Sending connection context
0811 06:30:19.225043 (+ 139us) client_negotiation.cc:240] Negotiation successful
0811 06:30:19.225450 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90}
I20260811 06:30:19.225890 8058 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81d907d1c7ce487da25ae51b57e094e3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c1478483ef29416fb130a1d621aafbaa" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c1478483ef29416fb130a1d621aafbaa" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 40129 } } peers { permanent_uuid: "81d907d1c7ce487da25ae51b57e094e3" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 45951 } attrs { promote: false } } }
I20260811 06:30:19.226173 8058 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81d907d1c7ce487da25ae51b57e094e3 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:30:19.226589 8187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:19.209564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55918 (local address 127.28.239.253:45951)
0811 06:30:19.210227 (+ 663us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:19.210231 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:19.210248 (+ 17us) server_negotiation.cc:407] Connection header received
0811 06:30:19.210292 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:19.210295 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:19.210352 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:19.210482 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:19.211275 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:19.212158 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:19.214046 (+ 1888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:19.214761 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:19.215562 (+ 801us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:19.218461 (+ 2899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:19.218466 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:19.218469 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:30:19.218502 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:19.222575 (+ 4073us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:19.223545 (+ 970us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:19.223555 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:19.223561 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:19.223721 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:19.224211 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:19.224217 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:19.224220 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:19.224597 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:19.224743 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:19.225222 (+ 479us) server_negotiation.cc:299] Negotiation successful
0811 06:30:19.225452 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":533,"thread_start_us":381,"threads_started":1}
I20260811 06:30:19.226421 8178 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1478483ef29416fb130a1d621aafbaa [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c1478483ef29416fb130a1d621aafbaa" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c1478483ef29416fb130a1d621aafbaa" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 40129 } } peers { permanent_uuid: "81d907d1c7ce487da25ae51b57e094e3" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 45951 } attrs { promote: false } } }
I20260811 06:30:19.226594 8180 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1478483ef29416fb130a1d621aafbaa [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c1478483ef29416fb130a1d621aafbaa" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c1478483ef29416fb130a1d621aafbaa" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 40129 } } peers { permanent_uuid: "81d907d1c7ce487da25ae51b57e094e3" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 45951 } attrs { promote: false } } }
I20260811 06:30:19.227224 8180 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1478483ef29416fb130a1d621aafbaa [sys.catalog]: This master's current role is: LEADER
I20260811 06:30:19.227473 8178 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1478483ef29416fb130a1d621aafbaa [sys.catalog]: This master's current role is: LEADER
I20260811 06:30:19.227705 8166 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.239.252" port: 43777 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 06:30:19 dist-test-slave-gkw7 krb5kdc[6988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429819, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.239.252@KRBTEST.COM for oryx/127.28.239.252@KRBTEST.COM
I20260811 06:30:19.240229 8173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:19.228416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.252:43777 (local address 127.0.0.1:45182)
0811 06:30:19.228628 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:19.228644 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:19.228777 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:19.229075 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:19.229079 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:19.229469 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:19.229675 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:19.229685 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:19.230623 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:19.230630 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:19.231473 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:19.231483 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:19.232186 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:19.232192 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:19.232384 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:19.232886 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:30:19.232901 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:19.234934 (+ 2033us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:30:19.237700 (+ 2766us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:30:19.237707 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:30:19.237711 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:30:19.238056 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:30:19.238440 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:30:19.238444 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:30:19.238448 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:30:19.238596 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:30:19.239197 (+ 601us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:30:19.239201 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:30:19.239340 (+ 139us) client_negotiation.cc:769] Sending connection context
0811 06:30:19.239478 (+ 138us) client_negotiation.cc:240] Negotiation successful
0811 06:30:19.239758 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77}
I20260811 06:30:19.240259 8172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:19.228441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45182 (local address 127.28.239.252:43777)
0811 06:30:19.228638 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:19.228644 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:19.228722 (+ 78us) server_negotiation.cc:407] Connection header received
0811 06:30:19.228891 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:19.228896 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:19.228956 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:19.229060 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:19.229803 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:19.230497 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:19.231612 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:19.232018 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:19.232149 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:19.235108 (+ 2959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:19.235114 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:19.235119 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:30:19.235160 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:19.237507 (+ 2347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:19.238183 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:19.238188 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:19.238192 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:19.238319 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:19.238716 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:19.238723 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:19.238726 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:19.239052 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:19.239200 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:19.239567 (+ 367us) server_negotiation.cc:299] Negotiation successful
0811 06:30:19.239912 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":126,"server-negotiator.queue_time_us":71}
I20260811 06:30:19.245280 8166 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3dbd88a041b4459980dab85a3d3bfaf7: Bootstrap starting.
I20260811 06:30:19.248548 8166 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3dbd88a041b4459980dab85a3d3bfaf7: Neither blocks nor log segments found. Creating new log.
I20260811 06:30:19.249626 8166 log.cc:824] T 00000000000000000000000000000000 P 3dbd88a041b4459980dab85a3d3bfaf7: Log is configured to *not* fsync() on all Append() calls
I20260811 06:30:19.252382 8166 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3dbd88a041b4459980dab85a3d3bfaf7: No bootstrap required, opened a new log
I20260811 06:30:19.260375 8166 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3dbd88a041b4459980dab85a3d3bfaf7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1478483ef29416fb130a1d621aafbaa" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 40129 } } peers { permanent_uuid: "81d907d1c7ce487da25ae51b57e094e3" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 45951 } } peers { permanent_uuid: "3dbd88a041b4459980dab85a3d3bfaf7" member_type: VOTER last_known_addr { host: "127.28.239.252" port: 43777 } }
I20260811 06:30:19.260907 8166 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3dbd88a041b4459980dab85a3d3bfaf7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:30:19.261042 8166 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3dbd88a041b4459980dab85a3d3bfaf7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3dbd88a041b4459980dab85a3d3bfaf7, State: Initialized, Role: FOLLOWER
I20260811 06:30:19.261839 8166 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3dbd88a041b4459980dab85a3d3bfaf7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1478483ef29416fb130a1d621aafbaa" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 40129 } } peers { permanent_uuid: "81d907d1c7ce487da25ae51b57e094e3" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 45951 } } peers { permanent_uuid: "3dbd88a041b4459980dab85a3d3bfaf7" member_type: VOTER last_known_addr { host: "127.28.239.252" port: 43777 } }
I20260811 06:30:19.262815 8192 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3dbd88a041b4459980dab85a3d3bfaf7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1478483ef29416fb130a1d621aafbaa" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 40129 } } peers { permanent_uuid: "81d907d1c7ce487da25ae51b57e094e3" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 45951 } } peers { permanent_uuid: "3dbd88a041b4459980dab85a3d3bfaf7" member_type: VOTER last_known_addr { host: "127.28.239.252" port: 43777 } } }
I20260811 06:30:19.263145 8192 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3dbd88a041b4459980dab85a3d3bfaf7 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:30:19.263931 8166 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3dbd88a041b4459980dab85a3d3bfaf7 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 06:30:19.273766 8212 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3dbd88a041b4459980dab85a3d3bfaf7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 06:30:19.273912 8212 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 06:30:19.279448 7936 master_runner.cc:428] Detected that this master 3dbd88a041b4459980dab85a3d3bfaf7 is joining an existing cluster
I20260811 06:30:19.279585 7936 master_runner.cc:432] Initiating AddMaster RPC to add 127.28.239.252:43777
I20260811 06:30:19.304396 7893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:19.284889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54072 (local address 127.28.239.254:40129)
0811 06:30:19.285182 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:19.285189 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:19.285209 (+ 20us) server_negotiation.cc:407] Connection header received
0811 06:30:19.285405 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:19.285410 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:19.285499 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:19.285611 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:19.287454 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:19.288927 (+ 1473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:19.291810 (+ 2883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:19.292728 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:19.292856 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:19.296590 (+ 3734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:19.296601 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:19.296609 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:30:19.296668 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:19.300617 (+ 3949us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:19.301454 (+ 837us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:19.301459 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:19.301463 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:19.301674 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:19.302178 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:19.302184 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:19.302188 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:19.302676 (+ 488us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:19.302830 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:19.303360 (+ 530us) server_negotiation.cc:299] Negotiation successful
0811 06:30:19.303593 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138}
I20260811 06:30:19.305378 8221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:19.284077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:40129 (local address 127.0.0.1:54072)
0811 06:30:19.284835 (+ 758us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:19.284856 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:19.285217 (+ 361us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:19.285852 (+ 635us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:19.285857 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:19.286718 (+ 861us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:19.287207 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:19.287225 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:19.289178 (+ 1953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:19.289183 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:19.291577 (+ 2394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:19.291600 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:19.293200 (+ 1600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:19.293208 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:19.293470 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:19.294436 (+ 966us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:30:19.294469 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:19.296298 (+ 1829us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:30:19.300768 (+ 4470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:30:19.300774 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:30:19.300778 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:30:19.301312 (+ 534us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:30:19.301820 (+ 508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:30:19.301827 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:30:19.301831 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:30:19.302049 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:30:19.302890 (+ 841us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:30:19.302893 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:30:19.303065 (+ 172us) client_negotiation.cc:769] Sending connection context
0811 06:30:19.303230 (+ 165us) client_negotiation.cc:240] Negotiation successful
0811 06:30:19.303587 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":596,"thread_start_us":1142,"threads_started":1}
I20260811 06:30:19.305668 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 7148
I20260811 06:30:19.313475 8172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:19.286011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45198 (local address 127.28.239.252:43777)
0811 06:30:19.286221 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:19.286228 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:19.298212 (+ 11984us) server_negotiation.cc:407] Connection header received
0811 06:30:19.298361 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:19.298367 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:19.298459 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:19.298574 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:19.299767 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:19.300906 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:19.302748 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:19.303443 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:19.303586 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:19.306387 (+ 2801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:19.306394 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:19.306399 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:30:19.306434 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:19.310331 (+ 3897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:19.311220 (+ 889us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:19.311230 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:19.311233 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:19.311388 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:19.311787 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:19.311792 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:19.311794 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:19.312158 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:19.312306 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:19.312890 (+ 584us) server_negotiation.cc:299] Negotiation successful
0811 06:30:19.313111 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 06:30:19.320996 8187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:19.284210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55920 (local address 127.28.239.253:45951)
0811 06:30:19.284400 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:19.284409 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:19.297347 (+ 12938us) server_negotiation.cc:407] Connection header received
0811 06:30:19.297646 (+ 299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:19.297654 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:19.297795 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:19.297882 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:19.304070 (+ 6188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:19.305296 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:19.307777 (+ 2481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:19.308521 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:19.308634 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:19.313996 (+ 5362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:19.314003 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:19.314007 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:30:19.314052 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:19.317666 (+ 3614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:19.318489 (+ 823us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:19.318495 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:19.318499 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:19.318668 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:19.319132 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:19.319139 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:19.319145 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:19.319575 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:19.319721 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:19.320355 (+ 634us) server_negotiation.cc:299] Negotiation successful
0811 06:30:19.320625 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
Aug 11 06:30:19 dist-test-slave-gkw7 krb5kdc[6988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429815, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.239.254@KRBTEST.COM for oryx/127.28.239.252@KRBTEST.COM
I20260811 06:30:19.325428 8223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:19.285702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.252:43777 (local address 127.0.0.1:45198)
0811 06:30:19.298084 (+ 12382us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:19.298105 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:19.298220 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:19.298637 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:19.298644 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:19.299225 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:19.299570 (+ 345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:19.299581 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:19.301077 (+ 1496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:19.301082 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:19.302583 (+ 1501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:19.302597 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:19.303879 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:19.303886 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:19.304118 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:19.304823 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:30:19.304844 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:19.306211 (+ 1367us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:30:19.310472 (+ 4261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:30:19.310478 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:30:19.310481 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:30:19.311085 (+ 604us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:30:19.311519 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:30:19.311523 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:30:19.311527 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:30:19.311679 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:30:19.312372 (+ 693us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:30:19.312377 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:30:19.312551 (+ 174us) client_negotiation.cc:769] Sending connection context
0811 06:30:19.324619 (+ 12068us) client_negotiation.cc:240] Negotiation successful
0811 06:30:19.324968 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12228,"thread_start_us":12075,"threads_started":1}
I20260811 06:30:19.325783 8222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:19.284607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.253:45951 (local address 127.0.0.1:55920)
0811 06:30:19.297032 (+ 12425us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:19.297050 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:19.297422 (+ 372us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:19.298141 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:19.298147 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:19.298934 (+ 787us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:19.299313 (+ 379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:19.299327 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:19.305796 (+ 6469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:19.305802 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:19.307521 (+ 1719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:19.307538 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:19.308951 (+ 1413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:19.308957 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:19.309220 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:19.309898 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:30:19.309925 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:19.311523 (+ 1598us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:30:19.317833 (+ 6310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:30:19.317838 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:30:19.317842 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:30:19.318328 (+ 486us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:30:19.318826 (+ 498us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:30:19.318832 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:30:19.318835 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:30:19.319011 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:30:19.319810 (+ 799us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:30:19.319815 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:30:19.320016 (+ 201us) client_negotiation.cc:769] Sending connection context
0811 06:30:19.321684 (+ 1668us) client_negotiation.cc:240] Negotiation successful
0811 06:30:19.322141 (+ 457us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12284,"mutex_wait_us":3,"thread_start_us":12034,"threads_started":1}
I20260811 06:30:19.333033 8182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:19.313661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.252:43777 (local address 127.0.0.1:45208)
0811 06:30:19.313858 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:19.313880 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:19.313994 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:19.314631 (+ 637us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:19.314647 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:19.315192 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:19.315399 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:19.315413 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:19.317055 (+ 1642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:19.317061 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:19.318705 (+ 1644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:19.318724 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:19.319774 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:19.319783 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:19.319972 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:19.320591 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:30:19.320615 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:19.323300 (+ 2685us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:30:19.329897 (+ 6597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:30:19.329904 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:30:19.329908 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:30:19.330502 (+ 594us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:30:19.330919 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:30:19.330924 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:30:19.330927 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:30:19.331135 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:30:19.331946 (+ 811us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:30:19.331951 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:30:19.332140 (+ 189us) client_negotiation.cc:769] Sending connection context
0811 06:30:19.332305 (+ 165us) client_negotiation.cc:240] Negotiation successful
0811 06:30:19.332660 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260811 06:30:19.333058 8172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:19.314083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45208 (local address 127.28.239.252:43777)
0811 06:30:19.314237 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:19.314242 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:19.314258 (+ 16us) server_negotiation.cc:407] Connection header received
0811 06:30:19.314371 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:19.314376 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:19.314457 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:19.315554 (+ 1097us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:19.315748 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:19.316885 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:19.318883 (+ 1998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:19.319575 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:19.323484 (+ 3909us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:19.323700 (+ 216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:19.323707 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:19.323711 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:30:19.323780 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:19.329722 (+ 5942us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:19.330624 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:19.330628 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:19.330631 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:19.330784 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:19.331298 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:19.331304 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:19.331307 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:19.331785 (+ 478us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:19.331956 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:19.332450 (+ 494us) server_negotiation.cc:299] Negotiation successful
0811 06:30:19.332695 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52,"thread_start_us":13411,"threads_started":1}
I20260811 06:30:19.334751 7048 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.28.239.252:43777: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "3dbd88a041b4459980dab85a3d3bfaf7"
member_type: NON_VOTER
last_known_addr {
host: "127.28.239.252"
port: 43777
}
attrs {
promote: true
}
}
dest_uuid: "c1478483ef29416fb130a1d621aafbaa"
cas_config_opid_index: 6
I20260811 06:30:19.335639 7048 consensus_queue.cc:237] T 00000000000000000000000000000000 P c1478483ef29416fb130a1d621aafbaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c1478483ef29416fb130a1d621aafbaa" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 40129 } } peers { permanent_uuid: "81d907d1c7ce487da25ae51b57e094e3" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 45951 } attrs { promote: false } } peers { permanent_uuid: "3dbd88a041b4459980dab85a3d3bfaf7" member_type: NON_VOTER last_known_addr { host: "127.28.239.252" port: 43777 } attrs { promote: true } }
Aug 11 06:30:19 dist-test-slave-gkw7 krb5kdc[6988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429815, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.28.239.193@KRBTEST.COM
I20260811 06:30:19.338042 8012 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 81d907d1c7ce487da25ae51b57e094e3 [term 1 FOLLOWER]: Refusing update from remote peer c1478483ef29416fb130a1d621aafbaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260811 06:30:19.338673 8178 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c1478483ef29416fb130a1d621aafbaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "81d907d1c7ce487da25ae51b57e094e3" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 45951 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260811 06:30:19.341741 8180 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c1478483ef29416fb130a1d621aafbaa [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 3dbd88a041b4459980dab85a3d3bfaf7 (127.28.239.252) added. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c1478483ef29416fb130a1d621aafbaa" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 40129 } } peers { permanent_uuid: "81d907d1c7ce487da25ae51b57e094e3" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 45951 } attrs { promote: false } } peers { permanent_uuid: "3dbd88a041b4459980dab85a3d3bfaf7" member_type: NON_VOTER last_known_addr { host: "127.28.239.252" port: 43777 } attrs { promote: true } } }
I20260811 06:30:19.345677 8180 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.28.239.252:43777
I20260811 06:30:19.345731 8012 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 81d907d1c7ce487da25ae51b57e094e3 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 3dbd88a041b4459980dab85a3d3bfaf7 (127.28.239.252) added. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c1478483ef29416fb130a1d621aafbaa" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 40129 } } peers { permanent_uuid: "81d907d1c7ce487da25ae51b57e094e3" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 45951 } attrs { promote: false } } peers { permanent_uuid: "3dbd88a041b4459980dab85a3d3bfaf7" member_type: NON_VOTER last_known_addr { host: "127.28.239.252" port: 43777 } attrs { promote: true } } }
I20260811 06:30:19.345999 8189 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1478483ef29416fb130a1d621aafbaa [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c1478483ef29416fb130a1d621aafbaa" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c1478483ef29416fb130a1d621aafbaa" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 40129 } } peers { permanent_uuid: "81d907d1c7ce487da25ae51b57e094e3" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 45951 } attrs { promote: false } } peers { permanent_uuid: "3dbd88a041b4459980dab85a3d3bfaf7" member_type: NON_VOTER last_known_addr { host: "127.28.239.252" port: 43777 } attrs { promote: true } } }
I20260811 06:30:19.346403 8189 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1478483ef29416fb130a1d621aafbaa [sys.catalog]: This master's current role is: LEADER
I20260811 06:30:19.346694 8180 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1478483ef29416fb130a1d621aafbaa [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c1478483ef29416fb130a1d621aafbaa" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c1478483ef29416fb130a1d621aafbaa" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 40129 } } peers { permanent_uuid: "81d907d1c7ce487da25ae51b57e094e3" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 45951 } attrs { promote: false } } peers { permanent_uuid: "3dbd88a041b4459980dab85a3d3bfaf7" member_type: NON_VOTER last_known_addr { host: "127.28.239.252" port: 43777 } attrs { promote: true } } }
I20260811 06:30:19.346963 8180 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1478483ef29416fb130a1d621aafbaa [sys.catalog]: This master's current role is: LEADER
I20260811 06:30:19.347050 8224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:19.307254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40570 (local address 127.28.239.193:40357)
0811 06:30:19.323530 (+ 16276us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:19.323542 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:19.323581 (+ 39us) server_negotiation.cc:407] Connection header received
0811 06:30:19.323675 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:19.323687 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:19.324038 (+ 351us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:19.326631 (+ 2593us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:19.327029 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:19.328857 (+ 1828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:19.330546 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:19.331386 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:19.331549 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:19.338258 (+ 6709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:19.338276 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:19.338297 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:30:19.338503 (+ 206us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:19.342621 (+ 4118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:19.343515 (+ 894us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:19.343525 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:19.343531 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:19.343700 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:19.344161 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:19.344166 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:19.344170 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:19.344593 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:19.344925 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:19.346022 (+ 1097us) server_negotiation.cc:299] Negotiation successful
0811 06:30:19.346452 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15643,"thread_start_us":15946,"threads_started":1}
I20260811 06:30:19.348909 8058 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81d907d1c7ce487da25ae51b57e094e3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c1478483ef29416fb130a1d621aafbaa" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c1478483ef29416fb130a1d621aafbaa" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 40129 } } peers { permanent_uuid: "81d907d1c7ce487da25ae51b57e094e3" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 45951 } attrs { promote: false } } peers { permanent_uuid: "3dbd88a041b4459980dab85a3d3bfaf7" member_type: NON_VOTER last_known_addr { host: "127.28.239.252" port: 43777 } attrs { promote: true } } }
I20260811 06:30:19.349213 8058 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81d907d1c7ce487da25ae51b57e094e3 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:30:19.350466 7936 master.cc:537] Master@127.28.239.252:43777 shutting down...
I20260811 06:30:19.350855 7351 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:30:19.352823 8134 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3dbd88a041b4459980dab85a3d3bfaf7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:30:19.358150 8134 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3dbd88a041b4459980dab85a3d3bfaf7 [term 1 FOLLOWER]: Refusing update from remote peer c1478483ef29416fb130a1d621aafbaa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260811 06:30:19.359553 8180 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c1478483ef29416fb130a1d621aafbaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "3dbd88a041b4459980dab85a3d3bfaf7" member_type: NON_VOTER last_known_addr { host: "127.28.239.252" port: 43777 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 06:30:19.363194 7027 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260811 06:30:19.370604 7027 consensus_peers.cc:487] T 00000000000000000000000000000000 P c1478483ef29416fb130a1d621aafbaa -> Peer 3dbd88a041b4459980dab85a3d3bfaf7 (127.28.239.252:43777): Couldn't send request to peer 3dbd88a041b4459980dab85a3d3bfaf7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260811 06:30:19.371196 7936 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 3dbd88a041b4459980dab85a3d3bfaf7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:30:19.371579 7936 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 3dbd88a041b4459980dab85a3d3bfaf7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:30:19.371783 7936 tablet_replica.cc:330] T 00000000000000000000000000000000 P 3dbd88a041b4459980dab85a3d3bfaf7: stopping tablet replica
I20260811 06:30:19.387408 7936 master.cc:552] Master@127.28.239.252:43777 shutdown complete.
I20260811 06:30:19.387523 7936 master_runner.cc:305] Clearing existing system tablet
I20260811 06:30:19.388002 7936 fs_manager.cc:359] Metadata directory not provided
I20260811 06:30:19.388132 7936 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:30:19.391873 7936 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 06:30:19.393707 8232 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:19.394059 7936 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:30:19.394502 7936 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 06:30:19.394680 7936 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/master-2/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/master-2/wal
uuid: "3dbd88a041b4459980dab85a3d3bfaf7"
format_stamp: "Formatted at 2026-08-11 06:30:19 on dist-test-slave-gkw7"
server_key: "89d1c39b4025c7532c0a8c10a8dc02f2"
server_key_iv: "6d2c865ef92a5233b9d348fbf9979e26"
server_key_version: "encryptionkey@0"
I20260811 06:30:19.400274 7936 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 3dbd88a041b4459980dab85a3d3bfaf7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:30:19.407606 7936 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 3dbd88a041b4459980dab85a3d3bfaf7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 06:30:19.407812 7936 log.cc:1196] T 00000000000000000000000000000000 P 3dbd88a041b4459980dab85a3d3bfaf7: Deleting WAL directory at /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260811 06:30:19.408396 7936 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 3dbd88a041b4459980dab85a3d3bfaf7: Deleting consensus metadata
I20260811 06:30:19.408753 7936 master_runner.cc:315] Copying system tablet from 127.28.239.254:40129
I20260811 06:30:19.418982 7936 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 3dbd88a041b4459980dab85a3d3bfaf7: tablet copy: Beginning tablet copy session from remote peer at address 127.28.239.254:40129
I20260811 06:30:19.444801 7893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:19.420068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54080 (local address 127.28.239.254:40129)
0811 06:30:19.420385 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:19.420395 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:19.426908 (+ 6513us) server_negotiation.cc:407] Connection header received
0811 06:30:19.427493 (+ 585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:19.427502 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:19.427639 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:19.427730 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:19.429319 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:19.430690 (+ 1371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:19.432958 (+ 2268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:19.433770 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:19.433894 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:19.436989 (+ 3095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:19.436999 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:19.437006 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:30:19.437059 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:19.441282 (+ 4223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:19.442356 (+ 1074us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:19.442364 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:19.442370 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:19.442544 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:19.443074 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:19.443079 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:19.443082 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:19.443445 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:19.443571 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:19.444170 (+ 599us) server_negotiation.cc:299] Negotiation successful
0811 06:30:19.444452 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112}
I20260811 06:30:19.445884 8241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:19.419990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:40129 (local address 127.0.0.1:54080)
0811 06:30:19.426741 (+ 6751us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:19.426768 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:19.427011 (+ 243us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:19.428024 (+ 1013us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:19.428035 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:19.428763 (+ 728us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:19.429106 (+ 343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:19.429122 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:19.430925 (+ 1803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:19.430933 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:19.432770 (+ 1837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:19.432789 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:19.434203 (+ 1414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:19.434213 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:19.434459 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:19.435236 (+ 777us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:30:19.435273 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:19.436719 (+ 1446us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:30:19.441513 (+ 4794us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:30:19.441521 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:30:19.441567 (+ 46us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:30:19.442189 (+ 622us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:30:19.442733 (+ 544us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:30:19.442740 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:30:19.442745 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:30:19.442944 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:30:19.443673 (+ 729us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:30:19.443679 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:30:19.443897 (+ 218us) client_negotiation.cc:769] Sending connection context
0811 06:30:19.445078 (+ 1181us) client_negotiation.cc:240] Negotiation successful
0811 06:30:19.445420 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6534,"thread_start_us":7059,"threads_started":1}
I20260811 06:30:19.446465 7080 tablet_copy_service.cc:140] P c1478483ef29416fb130a1d621aafbaa: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 3dbd88a041b4459980dab85a3d3bfaf7 ({username='oryx', principal='oryx/127.28.239.252@KRBTEST.COM'} at 127.0.0.1:54080)
I20260811 06:30:19.446661 7080 tablet_copy_service.cc:161] P c1478483ef29416fb130a1d621aafbaa: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 3dbd88a041b4459980dab85a3d3bfaf7 at {username='oryx', principal='oryx/127.28.239.252@KRBTEST.COM'} at 127.0.0.1:54080: session id = 3dbd88a041b4459980dab85a3d3bfaf7-00000000000000000000000000000000
I20260811 06:30:19.450174 7080 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P c1478483ef29416fb130a1d621aafbaa: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 06:30:19.452107 7936 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 06:30:19.459102 7936 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 3dbd88a041b4459980dab85a3d3bfaf7: tablet copy: Starting download of 0 data blocks...
I20260811 06:30:19.459414 7936 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 3dbd88a041b4459980dab85a3d3bfaf7: tablet copy: Starting download of 1 WAL segments...
I20260811 06:30:19.461879 7936 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 3dbd88a041b4459980dab85a3d3bfaf7: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 06:30:19.466125 7080 tablet_copy_service.cc:342] P c1478483ef29416fb130a1d621aafbaa: Request end of tablet copy session 3dbd88a041b4459980dab85a3d3bfaf7-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.28.239.252@KRBTEST.COM'} at 127.0.0.1:54080
I20260811 06:30:19.466281 7080 tablet_copy_service.cc:433] P c1478483ef29416fb130a1d621aafbaa: ending tablet copy session 3dbd88a041b4459980dab85a3d3bfaf7-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 3dbd88a041b4459980dab85a3d3bfaf7
I20260811 06:30:19.480026 7936 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:30:19.483421 8243 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:30:19.487244 8248 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:30:19.501899 7936 server_base.cc:1034] running on GCE node
W20260811 06:30:19.503625 8251 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:30:19.504175 7936 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:30:19.504988 7936 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:30:19.506244 7936 hybrid_clock.cc:648] HybridClock initialized: now 1786429819506207 us; error 41 us; skew 500 ppm
I20260811 06:30:19.506503 7936 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:30:19 dist-test-slave-gkw7 krb5kdc[6988](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429819, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.239.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:30:19.508841 7936 init.cc:377] Logged in from keytab as oryx/127.28.239.252@KRBTEST.COM (short username oryx)
I20260811 06:30:19.513841 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 7148
I20260811 06:30:19.517452 7936 webserver.cc:460] Webserver started at http://127.28.239.252:39295/ using document root <none> and password file <none>
I20260811 06:30:19.517841 7936 fs_manager.cc:359] Metadata directory not provided
I20260811 06:30:19.517952 7936 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:30:19.521339 7936 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260811 06:30:19.523375 8260 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:19.523873 7936 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 06:30:19.524050 7936 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/master-2/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/master-2/wal
uuid: "3dbd88a041b4459980dab85a3d3bfaf7"
format_stamp: "Formatted at 2026-08-11 06:30:19 on dist-test-slave-gkw7"
server_key: "89d1c39b4025c7532c0a8c10a8dc02f2"
server_key_iv: "6d2c865ef92a5233b9d348fbf9979e26"
server_key_version: "encryptionkey@0"
I20260811 06:30:19.524366 7936 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:30:19.538887 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.193:40357
--local_ip_for_outbound_sockets=127.28.239.193
--tserver_master_addrs=127.28.239.254:40129
--webserver_port=32959
--webserver_interface=127.28.239.193
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/krb5kdc/oryx_127.28.239.193.keytab
--principal=oryx/127.28.239.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:43555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.28.239.254:40129,127.28.239.253:45951
--tserver_master_addrs=127.28.239.254:40129,127.28.239.253:45951,127.28.239.252:43777 with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 06:30:19.545450 7936 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:30:19.549705 7936 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:30:19.580121 7936 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.252:43777
I20260811 06:30:19.580183 8342 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.252:43777 every 8 connection(s)
I20260811 06:30:19.581300 7936 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/master-2/data/info.pb
I20260811 06:30:19.587738 8343 sys_catalog.cc:263] Verifying existing consensus state
I20260811 06:30:19.591429 8343 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3dbd88a041b4459980dab85a3d3bfaf7: Bootstrap starting.
I20260811 06:30:19.612430 8343 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3dbd88a041b4459980dab85a3d3bfaf7: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 06:30:19.613014 8343 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3dbd88a041b4459980dab85a3d3bfaf7: Bootstrap complete.
I20260811 06:30:19.614230 8343 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3dbd88a041b4459980dab85a3d3bfaf7 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c1478483ef29416fb130a1d621aafbaa" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 40129 } } peers { permanent_uuid: "81d907d1c7ce487da25ae51b57e094e3" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 45951 } attrs { promote: false } } peers { permanent_uuid: "3dbd88a041b4459980dab85a3d3bfaf7" member_type: NON_VOTER last_known_addr { host: "127.28.239.252" port: 43777 } attrs { promote: true } }
I20260811 06:30:19.614554 8343 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3dbd88a041b4459980dab85a3d3bfaf7 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 3dbd88a041b4459980dab85a3d3bfaf7, State: Initialized, Role: LEARNER
I20260811 06:30:19.614821 8343 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3dbd88a041b4459980dab85a3d3bfaf7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c1478483ef29416fb130a1d621aafbaa" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 40129 } } peers { permanent_uuid: "81d907d1c7ce487da25ae51b57e094e3" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 45951 } attrs { promote: false } } peers { permanent_uuid: "3dbd88a041b4459980dab85a3d3bfaf7" member_type: NON_VOTER last_known_addr { host: "127.28.239.252" port: 43777 } attrs { promote: true } }
I20260811 06:30:19.615757 8346 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3dbd88a041b4459980dab85a3d3bfaf7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c1478483ef29416fb130a1d621aafbaa" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 40129 } } peers { permanent_uuid: "81d907d1c7ce487da25ae51b57e094e3" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 45951 } attrs { promote: false } } peers { permanent_uuid: "3dbd88a041b4459980dab85a3d3bfaf7" member_type: NON_VOTER last_known_addr { host: "127.28.239.252" port: 43777 } attrs { promote: true } } }
I20260811 06:30:19.616104 8343 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3dbd88a041b4459980dab85a3d3bfaf7 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:30:19.616295 8346 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3dbd88a041b4459980dab85a3d3bfaf7 [sys.catalog]: This master's current role is: LEARNER
I20260811 06:30:19.627547 8365 catalog_manager.cc:1259] Loaded cluster ID: e6f352fda9a24973ae7bc5beafb7d908
I20260811 06:30:19.627663 8365 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3dbd88a041b4459980dab85a3d3bfaf7: loading cluster ID for follower catalog manager: success
I20260811 06:30:19.631000 8365 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3dbd88a041b4459980dab85a3d3bfaf7: acquiring CA information for follower catalog manager: success
I20260811 06:30:19.633651 8365 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3dbd88a041b4459980dab85a3d3bfaf7: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 06:30:19.805231 8182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:19.797922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.252:43777 (local address 127.0.0.1:45216)
0811 06:30:19.798177 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:19.798205 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:19.798386 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:19.799121 (+ 735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:19.799128 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:19.799178 (+ 50us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:30:19.799391 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:19.799405 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:19.800784 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:19.800790 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:19.802905 (+ 2115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:19.802922 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:19.804308 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:19.804314 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:19.804566 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:19.804604 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 06:30:19.804709 (+ 105us) client_negotiation.cc:240] Negotiation successful
0811 06:30:19.804865 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74}
I20260811 06:30:19.805255 8366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:19.798151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45216 (local address 127.28.239.252:43777)
0811 06:30:19.798754 (+ 603us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:19.798758 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:19.798780 (+ 22us) server_negotiation.cc:407] Connection header received
0811 06:30:19.798839 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:19.798843 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:19.798908 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:19.799036 (+ 128us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:30:19.799557 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:19.800623 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:19.803139 (+ 2516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:19.804101 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:19.804295 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:19.804548 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:19.804825 (+ 277us) server_negotiation.cc:299] Negotiation successful
0811 06:30:19.804943 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":501,"thread_start_us":343,"threads_started":1}
I20260811 06:30:19.806581 8304 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 3dbd88a041b4459980dab85a3d3bfaf7 [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.7] Dedup: 1.7->[]
I20260811 06:30:19.807299 8346 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3dbd88a041b4459980dab85a3d3bfaf7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c1478483ef29416fb130a1d621aafbaa. Latest consensus state: current_term: 1 leader_uuid: "c1478483ef29416fb130a1d621aafbaa" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c1478483ef29416fb130a1d621aafbaa" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 40129 } } peers { permanent_uuid: "81d907d1c7ce487da25ae51b57e094e3" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 45951 } attrs { promote: false } } peers { permanent_uuid: "3dbd88a041b4459980dab85a3d3bfaf7" member_type: NON_VOTER last_known_addr { host: "127.28.239.252" port: 43777 } attrs { promote: true } } }
I20260811 06:30:19.807670 8346 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3dbd88a041b4459980dab85a3d3bfaf7 [sys.catalog]: This master's current role is: LEARNER
W20260811 06:30:19.922952 8266 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:30:19.923273 8266 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:30:19.923444 8266 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:30:19.932260 8266 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:30:19.932507 8266 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:30:19.932761 8266 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.193
I20260811 06:30:19.947036 8266 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:43555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/krb5kdc/oryx_127.28.239.193.keytab
--principal=oryx/127.28.239.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.193:40357
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.239.193
--webserver_port=32959
--tserver_master_addrs=127.28.239.254:40129,127.28.239.253:45951,127.28.239.252:43777
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.193
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:30:19.948676 8266 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:30:19.950261 8266 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:30:19.968643 8372 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:30:19.969018 8374 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:30:19.969519 8377 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:30:19.970917 8266 server_base.cc:1034] running on GCE node
I20260811 06:30:19.971549 8266 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:30:19.972715 8266 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:30:19.974025 8266 hybrid_clock.cc:648] HybridClock initialized: now 1786429819973888 us; error 59 us; skew 500 ppm
I20260811 06:30:19.974545 8266 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:30:19 dist-test-slave-gkw7 krb5kdc[6988](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429819, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.239.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:30:19.979696 8266 init.cc:377] Logged in from keytab as oryx/127.28.239.193@KRBTEST.COM (short username oryx)
I20260811 06:30:19.982638 8266 webserver.cc:460] Webserver started at http://127.28.239.193:32959/ using document root <none> and password file <none>
I20260811 06:30:19.983590 8266 fs_manager.cc:359] Metadata directory not provided
I20260811 06:30:19.983805 8266 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:30:19.990314 8266 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260811 06:30:19.993307 8386 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:19.994890 8266 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:30:19.995086 8266 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-0/wal
uuid: "0b83ec152b074f80af2848108d3af254"
format_stamp: "Formatted at 2026-08-11 06:30:16 on dist-test-slave-gkw7"
server_key: "bda6ace95f993a242a367028f6c5508a"
server_key_iv: "e06483c3ac7fa86cc1b08bc828c3a7cd"
server_key_version: "encryptionkey@0"
I20260811 06:30:19.996218 8266 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:30:20.016703 8266 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:30:20.022536 8266 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:30:20.022861 8266 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:30:20.024183 8266 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:30:20.026899 8266 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:30:20.027011 8266 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:20.027104 8266 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:30:20.027280 8266 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:20.068497 8266 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.193:40357
I20260811 06:30:20.068599 8564 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.193:40357 every 8 connection(s)
I20260811 06:30:20.070276 8266 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
I20260811 06:30:20.079591 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 8266
I20260811 06:30:20.080080 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 7363
Aug 11 06:30:20 dist-test-slave-gkw7 krb5kdc[6988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429819, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.239.193@KRBTEST.COM for oryx/127.28.239.252@KRBTEST.COM
Aug 11 06:30:20 dist-test-slave-gkw7 krb5kdc[6988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429819, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.239.193@KRBTEST.COM for oryx/127.28.239.253@KRBTEST.COM
I20260811 06:30:20.108806 8366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:20.073955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.193:32917 (local address 127.28.239.252:43777)
0811 06:30:20.074238 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:20.074248 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:20.084263 (+ 10015us) server_negotiation.cc:407] Connection header received
0811 06:30:20.084539 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:20.084545 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:20.084655 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:20.084744 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:20.086886 (+ 2142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:20.088202 (+ 1316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:20.092189 (+ 3987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:20.092910 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:20.093032 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:20.099318 (+ 6286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:20.099326 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:20.099332 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:30:20.099380 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:20.103972 (+ 4592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:20.105331 (+ 1359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:20.105337 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:20.105340 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:20.105518 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:20.106515 (+ 997us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:20.106521 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:20.106524 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:20.106919 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:20.107087 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:20.108161 (+ 1074us) server_negotiation.cc:299] Negotiation successful
0811 06:30:20.108433 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260811 06:30:20.109190 8570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:20.073765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.193:53645 (local address 127.28.239.253:45951)
0811 06:30:20.080378 (+ 6613us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:20.080386 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:20.080423 (+ 37us) server_negotiation.cc:407] Connection header received
0811 06:30:20.083515 (+ 3092us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:20.083522 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:20.083634 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:20.083800 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:20.087264 (+ 3464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:20.088568 (+ 1304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:20.091834 (+ 3266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:20.092567 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:20.092722 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:20.100233 (+ 7511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:20.100244 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:20.100252 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:30:20.100303 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:20.104580 (+ 4277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:20.105559 (+ 979us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:20.105565 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:20.105569 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:20.105753 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:20.106445 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:20.106452 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:20.106458 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:20.107227 (+ 769us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:20.107372 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:20.108474 (+ 1102us) server_negotiation.cc:299] Negotiation successful
0811 06:30:20.108766 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6470,"thread_start_us":6395,"threads_started":1}
I20260811 06:30:20.111068 8571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:20.074778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.253:45951 (local address 127.28.239.193:53645)
0811 06:30:20.076277 (+ 1499us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:20.076343 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:20.083180 (+ 6837us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:20.084374 (+ 1194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:20.084388 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:20.085591 (+ 1203us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:20.087064 (+ 1473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:20.087075 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:20.088768 (+ 1693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:20.088780 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:20.091370 (+ 2590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:20.091386 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:20.093107 (+ 1721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:20.093129 (+ 22us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:20.093506 (+ 377us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:20.094670 (+ 1164us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:30:20.094695 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:20.099934 (+ 5239us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:30:20.104784 (+ 4850us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:30:20.104789 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:30:20.104794 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:30:20.105378 (+ 584us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:30:20.105915 (+ 537us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:30:20.105921 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:30:20.105924 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:30:20.106278 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:30:20.107378 (+ 1100us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:30:20.107387 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:30:20.107761 (+ 374us) client_negotiation.cc:769] Sending connection context
0811 06:30:20.108008 (+ 247us) client_negotiation.cc:240] Negotiation successful
0811 06:30:20.109079 (+ 1071us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1001,"mutex_wait_us":1433,"spinlock_wait_cycles":26496,"thread_start_us":986,"threads_started":1}
I20260811 06:30:20.111110 8572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:20.075601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.252:43777 (local address 127.28.239.193:32917)
0811 06:30:20.084144 (+ 8543us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:20.084165 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:20.084374 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:20.085020 (+ 646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:20.085025 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:20.085768 (+ 743us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:20.086678 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:20.086703 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:20.088424 (+ 1721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:20.088430 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:20.091988 (+ 3558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:20.092003 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:20.093385 (+ 1382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:20.093392 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:20.093590 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:20.094589 (+ 999us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:30:20.094639 (+ 50us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:20.099011 (+ 4372us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:30:20.104221 (+ 5210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:30:20.104233 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:30:20.104258 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:30:20.105117 (+ 859us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:30:20.106148 (+ 1031us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:30:20.106153 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:30:20.106156 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:30:20.106365 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:30:20.107216 (+ 851us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:30:20.107225 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:30:20.107695 (+ 470us) client_negotiation.cc:769] Sending connection context
0811 06:30:20.108022 (+ 327us) client_negotiation.cc:240] Negotiation successful
0811 06:30:20.108649 (+ 627us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8375,"mutex_wait_us":133,"spinlock_wait_cycles":6784,"thread_start_us":8175,"threads_started":1}
Aug 11 06:30:20 dist-test-slave-gkw7 krb5kdc[6988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429819, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.239.193@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM
I20260811 06:30:20.113919 8568 heartbeater.cc:347] Connected to a master server at 127.28.239.253:45951
I20260811 06:30:20.114364 8568 heartbeater.cc:464] Registering TS with master...
I20260811 06:30:20.114670 8565 heartbeater.cc:347] Connected to a master server at 127.28.239.252:43777
I20260811 06:30:20.114831 8565 heartbeater.cc:464] Registering TS with master...
I20260811 06:30:20.115619 8568 heartbeater.cc:511] Master 127.28.239.253:45951 requested a full tablet report, sending...
I20260811 06:30:20.118503 7994 ts_manager.cc:194] Registered new tserver with Master: 0b83ec152b074f80af2848108d3af254 (127.28.239.193:40357)
I20260811 06:30:20.120150 8565 heartbeater.cc:511] Master 127.28.239.252:43777 requested a full tablet report, sending...
Aug 11 06:30:20 dist-test-slave-gkw7 krb5kdc[6988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429815, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.28.239.194@KRBTEST.COM
I20260811 06:30:20.122592 8285 ts_manager.cc:194] Registered new tserver with Master: 0b83ec152b074f80af2848108d3af254 (127.28.239.193:40357)
I20260811 06:30:20.124123 8577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:20.089869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:40129 (local address 127.28.239.193:33763)
0811 06:30:20.091007 (+ 1138us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:20.091029 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:20.094866 (+ 3837us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:20.097033 (+ 2167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:20.097042 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:20.097887 (+ 845us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:20.098496 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:20.098518 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:20.101335 (+ 2817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:20.101344 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:20.103826 (+ 2482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:20.103849 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:20.106385 (+ 2536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:20.106393 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:20.106594 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:20.109697 (+ 3103us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:30:20.109722 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:20.113910 (+ 4188us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:30:20.119256 (+ 5346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:30:20.119261 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:30:20.119265 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:30:20.119984 (+ 719us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:30:20.120610 (+ 626us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:30:20.120616 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:30:20.120620 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:30:20.120876 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:30:20.121640 (+ 764us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:30:20.121645 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:30:20.121837 (+ 192us) client_negotiation.cc:769] Sending connection context
0811 06:30:20.123180 (+ 1343us) client_negotiation.cc:240] Negotiation successful
0811 06:30:20.123621 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":994,"thread_start_us":757,"threads_started":1}
I20260811 06:30:20.123090 8576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:20.084612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.193:33763 (local address 127.28.239.254:40129)
0811 06:30:20.096085 (+ 11473us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:20.096095 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:20.096320 (+ 225us) server_negotiation.cc:407] Connection header received
0811 06:30:20.096403 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:20.096412 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:20.096542 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:20.096722 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:20.098771 (+ 2049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:20.101104 (+ 2333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:20.104078 (+ 2974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:20.105249 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:20.105489 (+ 240us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:20.114304 (+ 8815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:20.114314 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:20.114320 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:30:20.114372 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:20.119029 (+ 4657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:20.120189 (+ 1160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:20.120196 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:20.120200 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:20.120438 (+ 238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:20.121009 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:20.121015 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:20.121019 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:20.121457 (+ 438us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:20.121659 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:20.122230 (+ 571us) server_negotiation.cc:299] Negotiation successful
0811 06:30:20.122560 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11276,"thread_start_us":11569,"threads_started":1}
I20260811 06:30:20.125344 8566 heartbeater.cc:347] Connected to a master server at 127.28.239.254:40129
I20260811 06:30:20.125509 8566 heartbeater.cc:464] Registering TS with master...
I20260811 06:30:20.126091 8566 heartbeater.cc:511] Master 127.28.239.254:40129 requested a full tablet report, sending...
I20260811 06:30:20.127532 7048 ts_manager.cc:194] Re-registered known tserver with Master: 0b83ec152b074f80af2848108d3af254 (127.28.239.193:40357)
I20260811 06:30:20.128742 7048 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.28.239.193@KRBTEST.COM'} at 127.28.239.193:33763
I20260811 06:30:20.130272 8575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:20.081693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33700 (local address 127.28.239.194:45127)
0811 06:30:20.106333 (+ 24640us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:20.106346 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:20.106381 (+ 35us) server_negotiation.cc:407] Connection header received
0811 06:30:20.106461 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:20.106472 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:20.106813 (+ 341us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:20.107006 (+ 193us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:20.109474 (+ 2468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:20.111272 (+ 1798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:20.113749 (+ 2477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:20.114780 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:20.115549 (+ 769us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:20.122136 (+ 6587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:20.122156 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:20.122179 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:30:20.122225 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:20.126612 (+ 4387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:20.127584 (+ 972us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:20.127596 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:20.127600 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:20.127793 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:20.128200 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:20.128205 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:20.128209 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:20.128676 (+ 467us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:20.129031 (+ 355us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:20.129454 (+ 423us) server_negotiation.cc:299] Negotiation successful
0811 06:30:20.129824 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24426,"thread_start_us":24204,"threads_started":1}
I20260811 06:30:20.131668 7552 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:30:20.211443 8180 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1478483ef29416fb130a1d621aafbaa [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c1478483ef29416fb130a1d621aafbaa" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c1478483ef29416fb130a1d621aafbaa" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 40129 } } peers { permanent_uuid: "81d907d1c7ce487da25ae51b57e094e3" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 45951 } attrs { promote: false } } peers { permanent_uuid: "3dbd88a041b4459980dab85a3d3bfaf7" member_type: NON_VOTER last_known_addr { host: "127.28.239.252" port: 43777 } attrs { promote: true } } }
I20260811 06:30:20.211942 8180 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1478483ef29416fb130a1d621aafbaa [sys.catalog]: This master's current role is: LEADER
I20260811 06:30:20.213208 8579 raft_consensus.cc:1050] T 00000000000000000000000000000000 P c1478483ef29416fb130a1d621aafbaa: attempting to promote NON_VOTER 3dbd88a041b4459980dab85a3d3bfaf7 to VOTER
I20260811 06:30:20.213907 8579 consensus_queue.cc:237] T 00000000000000000000000000000000 P c1478483ef29416fb130a1d621aafbaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c1478483ef29416fb130a1d621aafbaa" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 40129 } } peers { permanent_uuid: "81d907d1c7ce487da25ae51b57e094e3" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 45951 } attrs { promote: false } } peers { permanent_uuid: "3dbd88a041b4459980dab85a3d3bfaf7" member_type: VOTER last_known_addr { host: "127.28.239.252" port: 43777 } attrs { promote: false } }
I20260811 06:30:20.215832 8304 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3dbd88a041b4459980dab85a3d3bfaf7 [term 1 LEARNER]: Refusing update from remote peer c1478483ef29416fb130a1d621aafbaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260811 06:30:20.215963 8012 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 81d907d1c7ce487da25ae51b57e094e3 [term 1 FOLLOWER]: Refusing update from remote peer c1478483ef29416fb130a1d621aafbaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260811 06:30:20.216502 8580 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c1478483ef29416fb130a1d621aafbaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "3dbd88a041b4459980dab85a3d3bfaf7" member_type: VOTER last_known_addr { host: "127.28.239.252" port: 43777 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260811 06:30:20.216982 8579 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c1478483ef29416fb130a1d621aafbaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "81d907d1c7ce487da25ae51b57e094e3" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 45951 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260811 06:30:20.220052 8180 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c1478483ef29416fb130a1d621aafbaa [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 3dbd88a041b4459980dab85a3d3bfaf7 (127.28.239.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c1478483ef29416fb130a1d621aafbaa" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 40129 } } peers { permanent_uuid: "81d907d1c7ce487da25ae51b57e094e3" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 45951 } attrs { promote: false } } peers { permanent_uuid: "3dbd88a041b4459980dab85a3d3bfaf7" member_type: VOTER last_known_addr { host: "127.28.239.252" port: 43777 } attrs { promote: false } } }
I20260811 06:30:20.220423 8012 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 81d907d1c7ce487da25ae51b57e094e3 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 3dbd88a041b4459980dab85a3d3bfaf7 (127.28.239.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c1478483ef29416fb130a1d621aafbaa" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 40129 } } peers { permanent_uuid: "81d907d1c7ce487da25ae51b57e094e3" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 45951 } attrs { promote: false } } peers { permanent_uuid: "3dbd88a041b4459980dab85a3d3bfaf7" member_type: VOTER last_known_addr { host: "127.28.239.252" port: 43777 } attrs { promote: false } } }
I20260811 06:30:20.224802 8180 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1478483ef29416fb130a1d621aafbaa [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c1478483ef29416fb130a1d621aafbaa" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c1478483ef29416fb130a1d621aafbaa" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 40129 } } peers { permanent_uuid: "81d907d1c7ce487da25ae51b57e094e3" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 45951 } attrs { promote: false } } peers { permanent_uuid: "3dbd88a041b4459980dab85a3d3bfaf7" member_type: VOTER last_known_addr { host: "127.28.239.252" port: 43777 } attrs { promote: false } } }
I20260811 06:30:20.225162 8180 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1478483ef29416fb130a1d621aafbaa [sys.catalog]: This master's current role is: LEADER
I20260811 06:30:20.225406 8180 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1478483ef29416fb130a1d621aafbaa [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c1478483ef29416fb130a1d621aafbaa" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c1478483ef29416fb130a1d621aafbaa" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 40129 } } peers { permanent_uuid: "81d907d1c7ce487da25ae51b57e094e3" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 45951 } attrs { promote: false } } peers { permanent_uuid: "3dbd88a041b4459980dab85a3d3bfaf7" member_type: VOTER last_known_addr { host: "127.28.239.252" port: 43777 } attrs { promote: false } } }
I20260811 06:30:20.225677 8180 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1478483ef29416fb130a1d621aafbaa [sys.catalog]: This master's current role is: LEADER
I20260811 06:30:20.226202 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 7363
I20260811 06:30:20.226612 8580 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1478483ef29416fb130a1d621aafbaa [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c1478483ef29416fb130a1d621aafbaa" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c1478483ef29416fb130a1d621aafbaa" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 40129 } } peers { permanent_uuid: "81d907d1c7ce487da25ae51b57e094e3" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 45951 } attrs { promote: false } } peers { permanent_uuid: "3dbd88a041b4459980dab85a3d3bfaf7" member_type: VOTER last_known_addr { host: "127.28.239.252" port: 43777 } attrs { promote: false } } }
I20260811 06:30:20.226855 8580 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1478483ef29416fb130a1d621aafbaa [sys.catalog]: This master's current role is: LEADER
I20260811 06:30:20.233158 8584 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81d907d1c7ce487da25ae51b57e094e3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c1478483ef29416fb130a1d621aafbaa" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c1478483ef29416fb130a1d621aafbaa" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 40129 } } peers { permanent_uuid: "81d907d1c7ce487da25ae51b57e094e3" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 45951 } attrs { promote: false } } peers { permanent_uuid: "3dbd88a041b4459980dab85a3d3bfaf7" member_type: VOTER last_known_addr { host: "127.28.239.252" port: 43777 } attrs { promote: false } } }
I20260811 06:30:20.233531 8584 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81d907d1c7ce487da25ae51b57e094e3 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:30:20.235843 8304 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 3dbd88a041b4459980dab85a3d3bfaf7 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 3dbd88a041b4459980dab85a3d3bfaf7 (127.28.239.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c1478483ef29416fb130a1d621aafbaa" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 40129 } } peers { permanent_uuid: "81d907d1c7ce487da25ae51b57e094e3" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 45951 } attrs { promote: false } } peers { permanent_uuid: "3dbd88a041b4459980dab85a3d3bfaf7" member_type: VOTER last_known_addr { host: "127.28.239.252" port: 43777 } attrs { promote: false } } }
I20260811 06:30:20.239216 8346 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3dbd88a041b4459980dab85a3d3bfaf7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c1478483ef29416fb130a1d621aafbaa" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c1478483ef29416fb130a1d621aafbaa" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 40129 } } peers { permanent_uuid: "81d907d1c7ce487da25ae51b57e094e3" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 45951 } attrs { promote: false } } peers { permanent_uuid: "3dbd88a041b4459980dab85a3d3bfaf7" member_type: VOTER last_known_addr { host: "127.28.239.252" port: 43777 } attrs { promote: false } } }
I20260811 06:30:20.239527 8346 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3dbd88a041b4459980dab85a3d3bfaf7 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:30:20.255657 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.194:45127
--local_ip_for_outbound_sockets=127.28.239.194
--tserver_master_addrs=127.28.239.254:40129
--webserver_port=36885
--webserver_interface=127.28.239.194
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/krb5kdc/oryx_127.28.239.194.keytab
--principal=oryx/127.28.239.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:43555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.28.239.254:40129,127.28.239.253:45951
--tserver_master_addrs=127.28.239.254:40129,127.28.239.253:45951,127.28.239.252:43777 with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:30:20.614131 8587 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:30:20.614454 8587 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:30:20.614553 8587 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:30:20.622367 8587 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:30:20.622491 8587 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:30:20.622606 8587 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.194
I20260811 06:30:20.634243 8587 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:43555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/krb5kdc/oryx_127.28.239.194.keytab
--principal=oryx/127.28.239.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.194:45127
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.239.194
--webserver_port=36885
--tserver_master_addrs=127.28.239.254:40129,127.28.239.253:45951,127.28.239.252:43777
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.194
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:30:20.635751 8587 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:30:20.637107 8587 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:30:20.651307 8598 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:30:20.651346 8596 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:30:20.651460 8595 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:30:20.652484 8587 server_base.cc:1034] running on GCE node
I20260811 06:30:20.653087 8587 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:30:20.654145 8587 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:30:20.655354 8587 hybrid_clock.cc:648] HybridClock initialized: now 1786429820655288 us; error 57 us; skew 500 ppm
I20260811 06:30:20.655642 8587 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:30:20 dist-test-slave-gkw7 krb5kdc[6988](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429820, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.239.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:30:20.658960 8587 init.cc:377] Logged in from keytab as oryx/127.28.239.194@KRBTEST.COM (short username oryx)
I20260811 06:30:20.660563 8587 webserver.cc:460] Webserver started at http://127.28.239.194:36885/ using document root <none> and password file <none>
I20260811 06:30:20.661242 8587 fs_manager.cc:359] Metadata directory not provided
I20260811 06:30:20.661365 8587 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:30:20.667157 8587 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 06:30:20.670089 8609 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:20.671538 8587 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260811 06:30:20.671736 8587 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-1/wal
uuid: "b8e3bd4d570d422d8dd77fd3c581232b"
format_stamp: "Formatted at 2026-08-11 06:30:17 on dist-test-slave-gkw7"
server_key: "767e61b7a3898b44f819d61c0d08734f"
server_key_iv: "d65dc91ea550030697e75b28026272de"
server_key_version: "encryptionkey@0"
I20260811 06:30:20.672786 8587 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:30:20.688710 8587 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:30:20.693280 8587 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:30:20.693673 8587 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:30:20.694855 8587 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:30:20.696329 8587 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:30:20.696458 8587 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:20.696588 8587 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:30:20.696695 8587 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:20.735658 8587 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.194:45127
I20260811 06:30:20.735726 8780 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.194:45127 every 8 connection(s)
I20260811 06:30:20.737206 8587 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
I20260811 06:30:20.743981 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 8587
I20260811 06:30:20.744398 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 7568
Aug 11 06:30:20 dist-test-slave-gkw7 krb5kdc[6988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429820, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.239.194@KRBTEST.COM for oryx/127.28.239.252@KRBTEST.COM
Aug 11 06:30:20 dist-test-slave-gkw7 krb5kdc[6988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429820, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.239.194@KRBTEST.COM for oryx/127.28.239.253@KRBTEST.COM
Aug 11 06:30:20 dist-test-slave-gkw7 krb5kdc[6988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429820, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.239.194@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM
Aug 11 06:30:20 dist-test-slave-gkw7 krb5kdc[6988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429815, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.28.239.195@KRBTEST.COM
I20260811 06:30:20.767599 8785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:20.740501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.194:59379 (local address 127.28.239.252:43777)
0811 06:30:20.741025 (+ 524us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:20.741032 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:20.742367 (+ 1335us) server_negotiation.cc:407] Connection header received
0811 06:30:20.746851 (+ 4484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:20.746858 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:20.746949 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:20.747033 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:20.750088 (+ 3055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:20.751208 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:20.753459 (+ 2251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:20.754107 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:20.754214 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:20.759843 (+ 5629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:20.759849 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:20.759853 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:30:20.759896 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:20.763512 (+ 3616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:20.764532 (+ 1020us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:20.764538 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:20.764541 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:20.764734 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:20.765304 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:20.765313 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:20.765316 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:20.765749 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:20.765884 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:20.766988 (+ 1104us) server_negotiation.cc:299] Negotiation successful
0811 06:30:20.767220 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396,"thread_start_us":283,"threads_started":1}
I20260811 06:30:20.769271 8790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:20.741836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.194:58393 (local address 127.28.239.253:45951)
0811 06:30:20.742972 (+ 1136us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:20.742979 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:20.743000 (+ 21us) server_negotiation.cc:407] Connection header received
0811 06:30:20.746960 (+ 3960us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:20.746966 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:20.747081 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:20.747191 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:20.750156 (+ 2965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:20.751242 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:20.753705 (+ 2463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:20.754373 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:20.754508 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:20.760134 (+ 5626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:20.760143 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:20.760151 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:30:20.760217 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:20.764106 (+ 3889us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:20.765059 (+ 953us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:20.765064 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:20.765068 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:20.765243 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:20.765708 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:20.765713 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:20.765716 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:20.766092 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:20.766302 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:20.768655 (+ 2353us) server_negotiation.cc:299] Negotiation successful
0811 06:30:20.768878 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1012,"thread_start_us":1188,"threads_started":1}
I20260811 06:30:20.769467 8792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:20.742881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:40129 (local address 127.28.239.194:48703)
0811 06:30:20.745386 (+ 2505us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:20.745407 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:20.746512 (+ 1105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:20.747453 (+ 941us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:20.747465 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:20.748586 (+ 1121us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:20.749866 (+ 1280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:20.749895 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:20.751396 (+ 1501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:20.751401 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:20.754378 (+ 2977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:20.754391 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:20.755630 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:20.755636 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:20.755859 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:20.756466 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:30:20.756501 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:20.761258 (+ 4757us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:30:20.765884 (+ 4626us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:30:20.765890 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:30:20.765893 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:30:20.766414 (+ 521us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:30:20.766937 (+ 523us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:30:20.766941 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:30:20.766945 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:30:20.767126 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:30:20.767867 (+ 741us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:30:20.767873 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:30:20.768041 (+ 168us) client_negotiation.cc:769] Sending connection context
0811 06:30:20.768198 (+ 157us) client_negotiation.cc:240] Negotiation successful
0811 06:30:20.768498 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2378,"mutex_wait_us":1419,"spinlock_wait_cycles":2816,"thread_start_us":2194,"threads_started":1}
I20260811 06:30:20.769732 8788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:20.741195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.253:45951 (local address 127.28.239.194:58393)
0811 06:30:20.742191 (+ 996us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:20.742242 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:20.746515 (+ 4273us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:20.747458 (+ 943us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:20.747467 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:20.748592 (+ 1125us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:20.749958 (+ 1366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:20.749974 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:20.751432 (+ 1458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:20.751437 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:20.753481 (+ 2044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:20.753496 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:20.754749 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:20.754755 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:20.754937 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:20.755827 (+ 890us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:30:20.755850 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:20.759808 (+ 3958us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:30:20.764299 (+ 4491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:30:20.764305 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:30:20.764309 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:30:20.764906 (+ 597us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:30:20.765377 (+ 471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:30:20.765383 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:30:20.765390 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:30:20.765566 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:30:20.766372 (+ 806us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:30:20.766378 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:30:20.766512 (+ 134us) client_negotiation.cc:769] Sending connection context
0811 06:30:20.766845 (+ 333us) client_negotiation.cc:240] Negotiation successful
0811 06:30:20.767421 (+ 576us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":783,"mutex_wait_us":226,"spinlock_wait_cycles":88448,"thread_start_us":489,"threads_started":1}
I20260811 06:30:20.770071 8786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:20.741195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.252:43777 (local address 127.28.239.194:59379)
0811 06:30:20.742191 (+ 996us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:20.742242 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:20.746512 (+ 4270us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:20.747453 (+ 941us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:20.747466 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:20.748576 (+ 1110us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:20.749866 (+ 1290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:20.749895 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:20.751396 (+ 1501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:20.751401 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:20.753291 (+ 1890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:20.753307 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:20.754562 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:20.754571 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:20.754893 (+ 322us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:20.755767 (+ 874us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:30:20.755812 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:20.759556 (+ 3744us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:30:20.763716 (+ 4160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:30:20.763725 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:30:20.763747 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:30:20.764384 (+ 637us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:30:20.764906 (+ 522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:30:20.764911 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:30:20.764914 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:30:20.765168 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:30:20.765989 (+ 821us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:30:20.765998 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:30:20.766509 (+ 511us) client_negotiation.cc:769] Sending connection context
0811 06:30:20.768529 (+ 2020us) client_negotiation.cc:240] Negotiation successful
0811 06:30:20.768854 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":638,"mutex_wait_us":271,"spinlock_wait_cycles":34048,"thread_start_us":664,"threads_started":1}
I20260811 06:30:20.771392 8800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:20.745791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43364 (local address 127.28.239.195:44989)
0811 06:30:20.746404 (+ 613us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:20.746410 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:20.746434 (+ 24us) server_negotiation.cc:407] Connection header received
0811 06:30:20.746521 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:20.746531 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:20.746816 (+ 285us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:20.746994 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:20.749502 (+ 2508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:20.751262 (+ 1760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:20.753211 (+ 1949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:20.754090 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:20.754215 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:20.762888 (+ 8673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:20.762907 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:20.762929 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:30:20.762981 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:20.766993 (+ 4012us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:20.767918 (+ 925us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:20.767927 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:20.767930 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:20.768097 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:20.768609 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:20.768615 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:20.768618 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:20.769070 (+ 452us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:20.769364 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:20.770691 (+ 1327us) server_negotiation.cc:299] Negotiation successful
0811 06:30:20.770979 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":461,"thread_start_us":365,"threads_started":1}
I20260811 06:30:20.771620 8783 heartbeater.cc:347] Connected to a master server at 127.28.239.253:45951
I20260811 06:30:20.771804 8796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:20.743002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.194:48703 (local address 127.28.239.254:40129)
0811 06:30:20.743493 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:20.743498 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:20.745518 (+ 2020us) server_negotiation.cc:407] Connection header received
0811 06:30:20.746934 (+ 1416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:20.746939 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:20.747023 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:20.747895 (+ 872us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:20.750087 (+ 2192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:20.751206 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:20.754543 (+ 3337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:20.755188 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:20.755327 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:20.761488 (+ 6161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:20.761493 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:20.761496 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:30:20.761577 (+ 81us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:20.765714 (+ 4137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:20.766595 (+ 881us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:20.766603 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:20.766607 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:20.766786 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:20.767262 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:20.767269 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:20.767271 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:20.767635 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:20.767797 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:20.771150 (+ 3353us) server_negotiation.cc:299] Negotiation successful
0811 06:30:20.771403 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":258,"threads_started":1}
I20260811 06:30:20.771961 8783 heartbeater.cc:464] Registering TS with master...
I20260811 06:30:20.772799 7756 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:30:20.772924 8783 heartbeater.cc:511] Master 127.28.239.253:45951 requested a full tablet report, sending...
I20260811 06:30:20.773118 8782 heartbeater.cc:347] Connected to a master server at 127.28.239.254:40129
I20260811 06:30:20.773293 8782 heartbeater.cc:464] Registering TS with master...
I20260811 06:30:20.773864 8782 heartbeater.cc:511] Master 127.28.239.254:40129 requested a full tablet report, sending...
I20260811 06:30:20.774107 8781 heartbeater.cc:347] Connected to a master server at 127.28.239.252:43777
I20260811 06:30:20.774283 8781 heartbeater.cc:464] Registering TS with master...
I20260811 06:30:20.774806 8781 heartbeater.cc:511] Master 127.28.239.252:43777 requested a full tablet report, sending...
I20260811 06:30:20.775092 7048 ts_manager.cc:194] Re-registered known tserver with Master: b8e3bd4d570d422d8dd77fd3c581232b (127.28.239.194:45127)
I20260811 06:30:20.776124 7048 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.28.239.194@KRBTEST.COM'} at 127.28.239.194:48703
I20260811 06:30:20.776352 7994 ts_manager.cc:194] Registered new tserver with Master: b8e3bd4d570d422d8dd77fd3c581232b (127.28.239.194:45127)
I20260811 06:30:20.776887 8285 ts_manager.cc:194] Registered new tserver with Master: b8e3bd4d570d422d8dd77fd3c581232b (127.28.239.194:45127)
I20260811 06:30:20.864732 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 7568
I20260811 06:30:20.887017 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.195:44989
--local_ip_for_outbound_sockets=127.28.239.195
--tserver_master_addrs=127.28.239.254:40129
--webserver_port=37013
--webserver_interface=127.28.239.195
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/krb5kdc/oryx_127.28.239.195.keytab
--principal=oryx/127.28.239.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:43555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.28.239.254:40129,127.28.239.253:45951
--tserver_master_addrs=127.28.239.254:40129,127.28.239.253:45951,127.28.239.252:43777 with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 06:30:21.131497 8566 heartbeater.cc:503] Master 127.28.239.254:40129 was elected leader, sending a full tablet report...
W20260811 06:30:21.242799 8802 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:30:21.243158 8802 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:30:21.243335 8802 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:30:21.250967 8802 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:30:21.251127 8802 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:30:21.251300 8802 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.195
I20260811 06:30:21.261183 8802 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:43555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/krb5kdc/oryx_127.28.239.195.keytab
--principal=oryx/127.28.239.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.195:44989
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.239.195
--webserver_port=37013
--tserver_master_addrs=127.28.239.254:40129,127.28.239.253:45951,127.28.239.252:43777
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.195
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:30:21.262710 8802 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:30:21.264111 8802 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:30:21.279461 8810 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:30:21.280270 8811 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:30:21.280335 8814 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:30:21.281260 8802 server_base.cc:1034] running on GCE node
I20260811 06:30:21.281967 8802 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:30:21.282914 8802 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:30:21.284166 8802 hybrid_clock.cc:648] HybridClock initialized: now 1786429821284130 us; error 59 us; skew 500 ppm
I20260811 06:30:21.284524 8802 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:30:21 dist-test-slave-gkw7 krb5kdc[6988](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429821, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.239.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:30:21.287890 8802 init.cc:377] Logged in from keytab as oryx/127.28.239.195@KRBTEST.COM (short username oryx)
I20260811 06:30:21.289331 8802 webserver.cc:460] Webserver started at http://127.28.239.195:37013/ using document root <none> and password file <none>
I20260811 06:30:21.290120 8802 fs_manager.cc:359] Metadata directory not provided
I20260811 06:30:21.290282 8802 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:30:21.295866 8802 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 06:30:21.299078 8822 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:21.300755 8802 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 06:30:21.300951 8802 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-2/wal
uuid: "7c0f7f24b8f0420a93c04bdf627b58e1"
format_stamp: "Formatted at 2026-08-11 06:30:17 on dist-test-slave-gkw7"
server_key: "7cdf79991a93a215d6c54600c142c9a2"
server_key_iv: "9bb7f1d7acb47984814262f2c3957857"
server_key_version: "encryptionkey@0"
I20260811 06:30:21.302073 8802 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:30:21.329715 8802 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:30:21.334684 8802 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:30:21.335000 8802 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:30:21.336073 8802 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:30:21.337653 8802 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:30:21.337776 8802 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:21.337898 8802 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:30:21.337996 8802 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:21.374100 8802 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.195:44989
I20260811 06:30:21.374166 8997 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.195:44989 every 8 connection(s)
I20260811 06:30:21.375666 8802 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
I20260811 06:30:21.384438 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 8802
Aug 11 06:30:21 dist-test-slave-gkw7 krb5kdc[6988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429821, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.239.195@KRBTEST.COM for oryx/127.28.239.252@KRBTEST.COM
Aug 11 06:30:21 dist-test-slave-gkw7 krb5kdc[6988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429821, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.239.195@KRBTEST.COM for oryx/127.28.239.253@KRBTEST.COM
Aug 11 06:30:21 dist-test-slave-gkw7 krb5kdc[6988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429821, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.239.195@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM
I20260811 06:30:21.403152 9003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:21.378944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.195:47481 (local address 127.28.239.252:43777)
0811 06:30:21.380334 (+ 1390us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:21.380342 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:21.381479 (+ 1137us) server_negotiation.cc:407] Connection header received
0811 06:30:21.383452 (+ 1973us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:21.383458 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:21.383543 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:21.383675 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:21.385825 (+ 2150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:21.386604 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:21.389044 (+ 2440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:21.389480 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:21.389659 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:21.394869 (+ 5210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:21.394874 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:21.394877 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:30:21.394910 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:21.398690 (+ 3780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:21.399649 (+ 959us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:21.399655 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:21.399658 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:21.399824 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:21.400932 (+ 1108us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:21.400937 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:21.400940 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:21.401317 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:21.401461 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:21.402526 (+ 1065us) server_negotiation.cc:299] Negotiation successful
0811 06:30:21.402798 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1245,"thread_start_us":1126,"threads_started":1}
I20260811 06:30:21.404960 9007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:21.379672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.252:43777 (local address 127.28.239.195:47481)
0811 06:30:21.380809 (+ 1137us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:21.380827 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:21.383166 (+ 2339us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:21.383927 (+ 761us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:21.383939 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:21.384885 (+ 946us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:21.385654 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:21.385675 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:21.386768 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:21.386774 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:21.388863 (+ 2089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:21.388875 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:21.389762 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:21.389771 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:21.390151 (+ 380us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:21.391287 (+ 1136us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:30:21.391326 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:21.394540 (+ 3214us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:30:21.398863 (+ 4323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:30:21.398873 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:30:21.398894 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:30:21.399484 (+ 590us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:30:21.399970 (+ 486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:30:21.399974 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:30:21.399976 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:30:21.400800 (+ 824us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:30:21.401587 (+ 787us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:30:21.401605 (+ 18us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:30:21.402083 (+ 478us) client_negotiation.cc:769] Sending connection context
0811 06:30:21.403680 (+ 1597us) client_negotiation.cc:240] Negotiation successful
0811 06:30:21.403987 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":976,"mutex_wait_us":469,"spinlock_wait_cycles":47872,"thread_start_us":765,"threads_started":1}
I20260811 06:30:21.407073 9009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:21.381437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:40129 (local address 127.28.239.195:55219)
0811 06:30:21.382127 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:21.382142 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:21.383190 (+ 1048us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:21.386190 (+ 3000us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:21.386196 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:21.386630 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:21.386903 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:21.386913 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:21.388299 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:21.388305 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:21.390031 (+ 1726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:21.390047 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:21.391445 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:21.391454 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:21.391678 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:21.392373 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:30:21.392394 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:21.396164 (+ 3770us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:30:21.401718 (+ 5554us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:30:21.401722 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:30:21.401725 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:30:21.402236 (+ 511us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:30:21.404897 (+ 2661us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:30:21.404904 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:30:21.404911 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:30:21.405170 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:30:21.406043 (+ 873us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:30:21.406047 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:30:21.406212 (+ 165us) client_negotiation.cc:769] Sending connection context
0811 06:30:21.406412 (+ 200us) client_negotiation.cc:240] Negotiation successful
0811 06:30:21.406696 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":572,"mutex_wait_us":49,"thread_start_us":418,"threads_started":1}
I20260811 06:30:21.407164 8998 heartbeater.cc:347] Connected to a master server at 127.28.239.252:43777
I20260811 06:30:21.407575 9002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:21.378838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.195:48391 (local address 127.28.239.253:45951)
0811 06:30:21.380124 (+ 1286us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:21.380132 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:21.380866 (+ 734us) server_negotiation.cc:407] Connection header received
0811 06:30:21.383553 (+ 2687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:21.383557 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:21.383627 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:21.383752 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:21.387057 (+ 3305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:21.387757 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:21.389954 (+ 2197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:21.390536 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:21.390670 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:21.395480 (+ 4810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:21.395485 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:21.395488 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:30:21.395552 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:21.399677 (+ 4125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:21.400628 (+ 951us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:21.400633 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:21.400636 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:21.400793 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:21.401309 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:21.401314 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:21.401317 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:21.402422 (+ 1105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:21.405356 (+ 2934us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:21.405493 (+ 137us) server_negotiation.cc:299] Negotiation successful
0811 06:30:21.405729 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1139,"thread_start_us":1026,"threads_started":1}
I20260811 06:30:21.407608 8998 heartbeater.cc:464] Registering TS with master...
I20260811 06:30:21.408732 8998 heartbeater.cc:511] Master 127.28.239.252:43777 requested a full tablet report, sending...
I20260811 06:30:21.410213 9010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:21.381628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.195:55219 (local address 127.28.239.254:40129)
0811 06:30:21.382481 (+ 853us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:21.382485 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:21.382500 (+ 15us) server_negotiation.cc:407] Connection header received
0811 06:30:21.383888 (+ 1388us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:21.383894 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:21.383974 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:21.384086 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:21.387057 (+ 2971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:21.388174 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:21.390196 (+ 2022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:21.390907 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:21.391066 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:21.396410 (+ 5344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:21.396420 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:21.396427 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:30:21.396487 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:21.401566 (+ 5079us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:21.402393 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:21.402401 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:21.402407 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:21.402611 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:21.405322 (+ 2711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:21.405331 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:21.405337 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:21.405909 (+ 572us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:21.406456 (+ 547us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:21.406641 (+ 185us) server_negotiation.cc:299] Negotiation successful
0811 06:30:21.406941 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":758,"thread_start_us":666,"threads_started":1}
I20260811 06:30:21.405047 9005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:21.379672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.253:45951 (local address 127.28.239.195:48391)
0811 06:30:21.380703 (+ 1031us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:21.380740 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:21.383190 (+ 2450us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:21.383929 (+ 739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:21.383939 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:21.384905 (+ 966us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:21.385654 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:21.385681 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:21.387904 (+ 2223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:21.387910 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:21.389715 (+ 1805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:21.389730 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:21.390736 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:21.390744 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:21.390878 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:21.391569 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:30:21.391611 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:21.395282 (+ 3671us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:30:21.399915 (+ 4633us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:30:21.399952 (+ 37us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:30:21.399957 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:30:21.400495 (+ 538us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:30:21.400927 (+ 432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:30:21.400931 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:30:21.401000 (+ 69us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:30:21.401156 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:30:21.402550 (+ 1394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:30:21.402554 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:30:21.402718 (+ 164us) client_negotiation.cc:769] Sending connection context
0811 06:30:21.402867 (+ 149us) client_negotiation.cc:240] Negotiation successful
0811 06:30:21.403425 (+ 558us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":751,"mutex_wait_us":7,"spinlock_wait_cycles":6784,"thread_start_us":447,"threads_started":1}
I20260811 06:30:21.411242 8285 ts_manager.cc:194] Registered new tserver with Master: 7c0f7f24b8f0420a93c04bdf627b58e1 (127.28.239.195:44989)
I20260811 06:30:21.411469 9000 heartbeater.cc:347] Connected to a master server at 127.28.239.253:45951
I20260811 06:30:21.411617 9000 heartbeater.cc:464] Registering TS with master...
I20260811 06:30:21.412122 9000 heartbeater.cc:511] Master 127.28.239.253:45951 requested a full tablet report, sending...
I20260811 06:30:21.412278 8999 heartbeater.cc:347] Connected to a master server at 127.28.239.254:40129
I20260811 06:30:21.412398 8999 heartbeater.cc:464] Registering TS with master...
I20260811 06:30:21.412896 8999 heartbeater.cc:511] Master 127.28.239.254:40129 requested a full tablet report, sending...
I20260811 06:30:21.413429 7994 ts_manager.cc:194] Registered new tserver with Master: 7c0f7f24b8f0420a93c04bdf627b58e1 (127.28.239.195:44989)
I20260811 06:30:21.413941 7048 ts_manager.cc:194] Re-registered known tserver with Master: 7c0f7f24b8f0420a93c04bdf627b58e1 (127.28.239.195:44989)
I20260811 06:30:21.415143 7048 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.28.239.195@KRBTEST.COM'} at 127.28.239.195:55219
I20260811 06:30:21.434617 9002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:21.412149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55924 (local address 127.28.239.253:45951)
0811 06:30:21.415712 (+ 3563us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:21.415721 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:21.415747 (+ 26us) server_negotiation.cc:407] Connection header received
0811 06:30:21.415818 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:21.415826 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:21.415990 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:21.416125 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:21.419104 (+ 2979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:21.420216 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:21.422800 (+ 2584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:21.423456 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:21.423607 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:21.427644 (+ 4037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:21.427651 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:21.427660 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:30:21.427693 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:21.431039 (+ 3346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:21.431811 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:21.431817 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:21.431820 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:21.431981 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:21.432446 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:21.432451 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:21.432454 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:21.432804 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:21.433012 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:21.433973 (+ 961us) server_negotiation.cc:299] Negotiation successful
0811 06:30:21.434235 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1056}
I20260811 06:30:21.434983 9010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:21.411696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54094 (local address 127.28.239.254:40129)
0811 06:30:21.411880 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:21.411888 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:21.418266 (+ 6378us) server_negotiation.cc:407] Connection header received
0811 06:30:21.418335 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:21.418342 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:21.418422 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:21.418533 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:21.421353 (+ 2820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:21.422184 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:21.424169 (+ 1985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:21.424712 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:21.424861 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:21.428282 (+ 3421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:21.428289 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:21.428293 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:30:21.428335 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:21.431519 (+ 3184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:21.432323 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:21.432329 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:21.432333 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:21.432518 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:21.433333 (+ 815us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:21.433339 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:21.433342 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:21.433711 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:21.433984 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:21.434400 (+ 416us) server_negotiation.cc:299] Negotiation successful
0811 06:30:21.434638 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 06:30:21.437479 9003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:21.415241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45228 (local address 127.28.239.252:43777)
0811 06:30:21.415421 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:21.415427 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:21.419904 (+ 4477us) server_negotiation.cc:407] Connection header received
0811 06:30:21.420049 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:21.420055 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:21.420195 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:21.420308 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:21.423033 (+ 2725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:21.423764 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:21.425837 (+ 2073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:21.426536 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:21.426689 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:21.430811 (+ 4122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:21.430820 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:21.430825 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:30:21.430868 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:21.434393 (+ 3525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:21.435243 (+ 850us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:21.435248 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:21.435250 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:21.435446 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:21.435885 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:21.435889 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:21.435892 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:21.436261 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:21.436531 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:21.436905 (+ 374us) server_negotiation.cc:299] Negotiation successful
0811 06:30:21.437154 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 06:30:21.439832 7048 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54094:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:30:21.446995 7048 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:30:21.491387 9044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:21.465175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40582 (local address 127.28.239.193:40357)
0811 06:30:21.482522 (+ 17347us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:21.482535 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:21.482573 (+ 38us) server_negotiation.cc:407] Connection header received
0811 06:30:21.482725 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:21.482735 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:21.483054 (+ 319us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:21.483317 (+ 263us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:30:21.483951 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:21.485423 (+ 1472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:21.488135 (+ 2712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:21.489740 (+ 1605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:21.489918 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:21.490128 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:21.490765 (+ 637us) server_negotiation.cc:299] Negotiation successful
0811 06:30:21.491059 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17184,"thread_start_us":17066,"threads_started":1}
I20260811 06:30:21.492323 9045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:21.464906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.193:40357 (local address 127.0.0.1:40582)
0811 06:30:21.477501 (+ 12595us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:21.477547 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:21.477717 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:21.483297 (+ 5580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:21.483306 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:21.483345 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:30:21.483756 (+ 411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:21.483772 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:21.485634 (+ 1862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:21.485643 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:21.487835 (+ 2192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:21.487853 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:21.489956 (+ 2103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:21.489962 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:21.490170 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:21.490495 (+ 325us) client_negotiation.cc:769] Sending connection context
0811 06:30:21.490604 (+ 109us) client_negotiation.cc:240] Negotiation successful
0811 06:30:21.491785 (+ 1181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12440,"mutex_wait_us":1293,"thread_start_us":17699,"threads_started":1}
I20260811 06:30:21.492374 9042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:21.465234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.194:45127 (local address 127.0.0.1:33712)
0811 06:30:21.482887 (+ 17653us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:21.482906 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:21.483180 (+ 274us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:21.484736 (+ 1556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:21.484740 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:21.484770 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:30:21.485221 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:21.485234 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:21.488046 (+ 2812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:21.488054 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:21.489676 (+ 1622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:21.489689 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:21.491527 (+ 1838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:21.491537 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:21.491735 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:21.491759 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 06:30:21.491814 (+ 55us) client_negotiation.cc:240] Negotiation successful
0811 06:30:21.491909 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17529,"mutex_wait_us":1,"thread_start_us":12604,"threads_started":1}
I20260811 06:30:21.493009 9046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:21.465596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33712 (local address 127.28.239.194:45127)
0811 06:30:21.482028 (+ 16432us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:21.482042 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:21.483057 (+ 1015us) server_negotiation.cc:407] Connection header received
0811 06:30:21.484063 (+ 1006us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:21.484089 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:21.484479 (+ 390us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:21.485418 (+ 939us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:30:21.486322 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:21.487868 (+ 1546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:21.489842 (+ 1974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:21.491269 (+ 1427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:21.492264 (+ 995us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:21.492469 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:21.492550 (+ 81us) server_negotiation.cc:299] Negotiation successful
0811 06:30:21.492726 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16268,"thread_start_us":16118,"threads_started":1}
I20260811 06:30:21.495322 9050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:21.482087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.195:44989 (local address 127.0.0.1:43374)
0811 06:30:21.485868 (+ 3781us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:21.485891 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:21.486041 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:21.486984 (+ 943us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:21.486989 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:21.487018 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:30:21.487448 (+ 430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:21.487460 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:21.489721 (+ 2261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:21.489725 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:21.492279 (+ 2554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:21.492294 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:21.494350 (+ 2056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:21.494356 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:21.494565 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:21.494589 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 06:30:21.494725 (+ 136us) client_negotiation.cc:240] Negotiation successful
0811 06:30:21.494881 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3568,"mutex_wait_us":4,"thread_start_us":2599,"threads_started":1}
I20260811 06:30:21.495492 9048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:21.475139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43374 (local address 127.28.239.195:44989)
0811 06:30:21.485261 (+ 10122us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:21.485269 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:21.485995 (+ 726us) server_negotiation.cc:407] Connection header received
0811 06:30:21.486220 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:21.486229 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:21.486488 (+ 259us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:21.486711 (+ 223us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:30:21.487681 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:21.489498 (+ 1817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:21.492481 (+ 2983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:21.494080 (+ 1599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:21.494265 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:21.494436 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:21.494953 (+ 517us) server_negotiation.cc:299] Negotiation successful
0811 06:30:21.495142 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9930,"thread_start_us":9926,"threads_started":1}
I20260811 06:30:21.496996 8460 tablet_service.cc:1467] Processing CreateTablet for tablet fdb20d7f3ada476096f320db6d088346 (DEFAULT_TABLE table=test-table [id=1a8119a454df42f29ad7b77c9d7d66bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:30:21.498668 8460 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fdb20d7f3ada476096f320db6d088346. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:30:21.499148 8678 tablet_service.cc:1467] Processing CreateTablet for tablet fdb20d7f3ada476096f320db6d088346 (DEFAULT_TABLE table=test-table [id=1a8119a454df42f29ad7b77c9d7d66bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:30:21.500530 8678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fdb20d7f3ada476096f320db6d088346. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:30:21.499346 8896 tablet_service.cc:1467] Processing CreateTablet for tablet fdb20d7f3ada476096f320db6d088346 (DEFAULT_TABLE table=test-table [id=1a8119a454df42f29ad7b77c9d7d66bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:30:21.501869 8896 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fdb20d7f3ada476096f320db6d088346. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:30:21.514178 9051 tablet_bootstrap.cc:492] T fdb20d7f3ada476096f320db6d088346 P b8e3bd4d570d422d8dd77fd3c581232b: Bootstrap starting.
I20260811 06:30:21.515724 9052 tablet_bootstrap.cc:492] T fdb20d7f3ada476096f320db6d088346 P 0b83ec152b074f80af2848108d3af254: Bootstrap starting.
I20260811 06:30:21.518267 9053 tablet_bootstrap.cc:492] T fdb20d7f3ada476096f320db6d088346 P 7c0f7f24b8f0420a93c04bdf627b58e1: Bootstrap starting.
I20260811 06:30:21.519752 9052 tablet_bootstrap.cc:654] T fdb20d7f3ada476096f320db6d088346 P 0b83ec152b074f80af2848108d3af254: Neither blocks nor log segments found. Creating new log.
I20260811 06:30:21.520299 9051 tablet_bootstrap.cc:654] T fdb20d7f3ada476096f320db6d088346 P b8e3bd4d570d422d8dd77fd3c581232b: Neither blocks nor log segments found. Creating new log.
I20260811 06:30:21.521185 9052 log.cc:824] T fdb20d7f3ada476096f320db6d088346 P 0b83ec152b074f80af2848108d3af254: Log is configured to *not* fsync() on all Append() calls
I20260811 06:30:21.521484 9053 tablet_bootstrap.cc:654] T fdb20d7f3ada476096f320db6d088346 P 7c0f7f24b8f0420a93c04bdf627b58e1: Neither blocks nor log segments found. Creating new log.
I20260811 06:30:21.521673 9051 log.cc:824] T fdb20d7f3ada476096f320db6d088346 P b8e3bd4d570d422d8dd77fd3c581232b: Log is configured to *not* fsync() on all Append() calls
I20260811 06:30:21.522944 9053 log.cc:824] T fdb20d7f3ada476096f320db6d088346 P 7c0f7f24b8f0420a93c04bdf627b58e1: Log is configured to *not* fsync() on all Append() calls
I20260811 06:30:21.524890 9052 tablet_bootstrap.cc:492] T fdb20d7f3ada476096f320db6d088346 P 0b83ec152b074f80af2848108d3af254: No bootstrap required, opened a new log
I20260811 06:30:21.525183 9052 ts_tablet_manager.cc:1397] T fdb20d7f3ada476096f320db6d088346 P 0b83ec152b074f80af2848108d3af254: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260811 06:30:21.525668 9051 tablet_bootstrap.cc:492] T fdb20d7f3ada476096f320db6d088346 P b8e3bd4d570d422d8dd77fd3c581232b: No bootstrap required, opened a new log
I20260811 06:30:21.526103 9051 ts_tablet_manager.cc:1397] T fdb20d7f3ada476096f320db6d088346 P b8e3bd4d570d422d8dd77fd3c581232b: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.003s
I20260811 06:30:21.526791 9053 tablet_bootstrap.cc:492] T fdb20d7f3ada476096f320db6d088346 P 7c0f7f24b8f0420a93c04bdf627b58e1: No bootstrap required, opened a new log
I20260811 06:30:21.527050 9053 ts_tablet_manager.cc:1397] T fdb20d7f3ada476096f320db6d088346 P 7c0f7f24b8f0420a93c04bdf627b58e1: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.003s
I20260811 06:30:21.533098 9052 raft_consensus.cc:348] T fdb20d7f3ada476096f320db6d088346 P 0b83ec152b074f80af2848108d3af254 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b83ec152b074f80af2848108d3af254" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 40357 } } peers { permanent_uuid: "b8e3bd4d570d422d8dd77fd3c581232b" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 45127 } } peers { permanent_uuid: "7c0f7f24b8f0420a93c04bdf627b58e1" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 44989 } }
I20260811 06:30:21.533512 9052 raft_consensus.cc:374] T fdb20d7f3ada476096f320db6d088346 P 0b83ec152b074f80af2848108d3af254 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:30:21.533660 9052 raft_consensus.cc:729] T fdb20d7f3ada476096f320db6d088346 P 0b83ec152b074f80af2848108d3af254 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b83ec152b074f80af2848108d3af254, State: Initialized, Role: FOLLOWER
I20260811 06:30:21.534322 9052 consensus_queue.cc:260] T fdb20d7f3ada476096f320db6d088346 P 0b83ec152b074f80af2848108d3af254 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b83ec152b074f80af2848108d3af254" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 40357 } } peers { permanent_uuid: "b8e3bd4d570d422d8dd77fd3c581232b" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 45127 } } peers { permanent_uuid: "7c0f7f24b8f0420a93c04bdf627b58e1" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 44989 } }
I20260811 06:30:21.537822 9053 raft_consensus.cc:348] T fdb20d7f3ada476096f320db6d088346 P 7c0f7f24b8f0420a93c04bdf627b58e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b83ec152b074f80af2848108d3af254" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 40357 } } peers { permanent_uuid: "b8e3bd4d570d422d8dd77fd3c581232b" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 45127 } } peers { permanent_uuid: "7c0f7f24b8f0420a93c04bdf627b58e1" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 44989 } }
I20260811 06:30:21.538340 9053 raft_consensus.cc:374] T fdb20d7f3ada476096f320db6d088346 P 7c0f7f24b8f0420a93c04bdf627b58e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:30:21.538456 9053 raft_consensus.cc:729] T fdb20d7f3ada476096f320db6d088346 P 7c0f7f24b8f0420a93c04bdf627b58e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c0f7f24b8f0420a93c04bdf627b58e1, State: Initialized, Role: FOLLOWER
I20260811 06:30:21.538340 9051 raft_consensus.cc:348] T fdb20d7f3ada476096f320db6d088346 P b8e3bd4d570d422d8dd77fd3c581232b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b83ec152b074f80af2848108d3af254" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 40357 } } peers { permanent_uuid: "b8e3bd4d570d422d8dd77fd3c581232b" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 45127 } } peers { permanent_uuid: "7c0f7f24b8f0420a93c04bdf627b58e1" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 44989 } }
I20260811 06:30:21.538877 9051 raft_consensus.cc:374] T fdb20d7f3ada476096f320db6d088346 P b8e3bd4d570d422d8dd77fd3c581232b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:30:21.538998 9051 raft_consensus.cc:729] T fdb20d7f3ada476096f320db6d088346 P b8e3bd4d570d422d8dd77fd3c581232b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e3bd4d570d422d8dd77fd3c581232b, State: Initialized, Role: FOLLOWER
I20260811 06:30:21.539101 9053 consensus_queue.cc:260] T fdb20d7f3ada476096f320db6d088346 P 7c0f7f24b8f0420a93c04bdf627b58e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b83ec152b074f80af2848108d3af254" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 40357 } } peers { permanent_uuid: "b8e3bd4d570d422d8dd77fd3c581232b" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 45127 } } peers { permanent_uuid: "7c0f7f24b8f0420a93c04bdf627b58e1" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 44989 } }
I20260811 06:30:21.539340 9052 ts_tablet_manager.cc:1428] T fdb20d7f3ada476096f320db6d088346 P 0b83ec152b074f80af2848108d3af254: Time spent starting tablet: real 0.014s user 0.007s sys 0.004s
I20260811 06:30:21.539741 9051 consensus_queue.cc:260] T fdb20d7f3ada476096f320db6d088346 P b8e3bd4d570d422d8dd77fd3c581232b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b83ec152b074f80af2848108d3af254" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 40357 } } peers { permanent_uuid: "b8e3bd4d570d422d8dd77fd3c581232b" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 45127 } } peers { permanent_uuid: "7c0f7f24b8f0420a93c04bdf627b58e1" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 44989 } }
I20260811 06:30:21.544920 8782 heartbeater.cc:503] Master 127.28.239.254:40129 was elected leader, sending a full tablet report...
I20260811 06:30:21.546932 8999 heartbeater.cc:503] Master 127.28.239.254:40129 was elected leader, sending a full tablet report...
I20260811 06:30:21.547523 9051 ts_tablet_manager.cc:1428] T fdb20d7f3ada476096f320db6d088346 P b8e3bd4d570d422d8dd77fd3c581232b: Time spent starting tablet: real 0.021s user 0.013s sys 0.004s
I20260811 06:30:21.549819 9053 ts_tablet_manager.cc:1428] T fdb20d7f3ada476096f320db6d088346 P 7c0f7f24b8f0420a93c04bdf627b58e1: Time spent starting tablet: real 0.023s user 0.014s sys 0.008s
W20260811 06:30:21.582739 8569 tablet.cc:2369] T fdb20d7f3ada476096f320db6d088346 P 0b83ec152b074f80af2848108d3af254: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 06:30:21.628162 9001 tablet.cc:2369] T fdb20d7f3ada476096f320db6d088346 P 7c0f7f24b8f0420a93c04bdf627b58e1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 06:30:21.741247 8784 tablet.cc:2369] T fdb20d7f3ada476096f320db6d088346 P b8e3bd4d570d422d8dd77fd3c581232b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:30:21.897087 9059 raft_consensus.cc:482] T fdb20d7f3ada476096f320db6d088346 P 7c0f7f24b8f0420a93c04bdf627b58e1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:30:21.897486 9059 raft_consensus.cc:504] T fdb20d7f3ada476096f320db6d088346 P 7c0f7f24b8f0420a93c04bdf627b58e1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b83ec152b074f80af2848108d3af254" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 40357 } } peers { permanent_uuid: "b8e3bd4d570d422d8dd77fd3c581232b" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 45127 } } peers { permanent_uuid: "7c0f7f24b8f0420a93c04bdf627b58e1" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 44989 } }
I20260811 06:30:21.899458 9059 leader_election.cc:288] T fdb20d7f3ada476096f320db6d088346 P 7c0f7f24b8f0420a93c04bdf627b58e1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0b83ec152b074f80af2848108d3af254 (127.28.239.193:40357), b8e3bd4d570d422d8dd77fd3c581232b (127.28.239.194:45127)
I20260811 06:30:21.905447 9044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:21.899811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.195:51325 (local address 127.28.239.193:40357)
0811 06:30:21.900031 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:21.900040 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:21.900172 (+ 132us) server_negotiation.cc:407] Connection header received
0811 06:30:21.900430 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:21.900438 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:21.900515 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:21.900657 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:30:21.901103 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:21.901917 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:21.903588 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:21.904445 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:21.904624 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:21.904704 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:21.905042 (+ 338us) server_negotiation.cc:299] Negotiation successful
0811 06:30:21.905177 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 06:30:21.905580 9046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:21.900210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.195:52509 (local address 127.28.239.194:45127)
0811 06:30:21.900468 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:21.900475 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:21.900496 (+ 21us) server_negotiation.cc:407] Connection header received
0811 06:30:21.900577 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:21.900584 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:21.900658 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:21.900761 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:30:21.901338 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:21.902129 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:21.903679 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:21.904469 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:21.904654 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:21.904736 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:21.905126 (+ 390us) server_negotiation.cc:299] Negotiation successful
0811 06:30:21.905264 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 06:30:21.905619 9009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:21.899946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.194:45127 (local address 127.28.239.195:52509)
0811 06:30:21.900210 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:21.900229 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:21.900343 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:21.900964 (+ 621us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:21.900972 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:21.901000 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:30:21.901160 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:21.901174 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:21.902276 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:21.902281 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:21.903503 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:21.903514 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:21.904666 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:21.904673 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:21.904950 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:21.904987 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 06:30:21.905098 (+ 111us) client_negotiation.cc:240] Negotiation successful
0811 06:30:21.905299 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"mutex_wait_us":49}
I20260811 06:30:21.905616 9005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:21.899855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.193:40357 (local address 127.28.239.195:51325)
0811 06:30:21.900048 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:21.900069 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:21.900229 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:21.900672 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:21.900677 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:21.900713 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:30:21.900926 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:21.900939 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:21.902073 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:21.902078 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:21.903418 (+ 1340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:21.903430 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:21.904610 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:21.904620 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:21.904875 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:21.904900 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 06:30:21.905015 (+ 115us) client_negotiation.cc:240] Negotiation successful
0811 06:30:21.905196 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54,"mutex_wait_us":1}
I20260811 06:30:21.906793 8712 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdb20d7f3ada476096f320db6d088346" candidate_uuid: "7c0f7f24b8f0420a93c04bdf627b58e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e3bd4d570d422d8dd77fd3c581232b" is_pre_election: true
I20260811 06:30:21.906837 8489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdb20d7f3ada476096f320db6d088346" candidate_uuid: "7c0f7f24b8f0420a93c04bdf627b58e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b83ec152b074f80af2848108d3af254" is_pre_election: true
I20260811 06:30:21.907366 8712 raft_consensus.cc:2454] T fdb20d7f3ada476096f320db6d088346 P b8e3bd4d570d422d8dd77fd3c581232b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c0f7f24b8f0420a93c04bdf627b58e1 in term 0.
I20260811 06:30:21.907423 8489 raft_consensus.cc:2454] T fdb20d7f3ada476096f320db6d088346 P 0b83ec152b074f80af2848108d3af254 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c0f7f24b8f0420a93c04bdf627b58e1 in term 0.
I20260811 06:30:21.908236 8827 leader_election.cc:302] T fdb20d7f3ada476096f320db6d088346 P 7c0f7f24b8f0420a93c04bdf627b58e1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c0f7f24b8f0420a93c04bdf627b58e1, b8e3bd4d570d422d8dd77fd3c581232b; no voters:
I20260811 06:30:21.908677 9059 raft_consensus.cc:2781] T fdb20d7f3ada476096f320db6d088346 P 7c0f7f24b8f0420a93c04bdf627b58e1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:30:21.908846 9059 raft_consensus.cc:482] T fdb20d7f3ada476096f320db6d088346 P 7c0f7f24b8f0420a93c04bdf627b58e1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:30:21.908978 9059 raft_consensus.cc:3037] T fdb20d7f3ada476096f320db6d088346 P 7c0f7f24b8f0420a93c04bdf627b58e1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:30:21.911376 9059 raft_consensus.cc:504] T fdb20d7f3ada476096f320db6d088346 P 7c0f7f24b8f0420a93c04bdf627b58e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b83ec152b074f80af2848108d3af254" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 40357 } } peers { permanent_uuid: "b8e3bd4d570d422d8dd77fd3c581232b" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 45127 } } peers { permanent_uuid: "7c0f7f24b8f0420a93c04bdf627b58e1" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 44989 } }
I20260811 06:30:21.911975 9059 leader_election.cc:288] T fdb20d7f3ada476096f320db6d088346 P 7c0f7f24b8f0420a93c04bdf627b58e1 [CANDIDATE]: Term 1 election: Requested vote from peers 0b83ec152b074f80af2848108d3af254 (127.28.239.193:40357), b8e3bd4d570d422d8dd77fd3c581232b (127.28.239.194:45127)
I20260811 06:30:21.912632 8712 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdb20d7f3ada476096f320db6d088346" candidate_uuid: "7c0f7f24b8f0420a93c04bdf627b58e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e3bd4d570d422d8dd77fd3c581232b"
I20260811 06:30:21.912652 8489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdb20d7f3ada476096f320db6d088346" candidate_uuid: "7c0f7f24b8f0420a93c04bdf627b58e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b83ec152b074f80af2848108d3af254"
I20260811 06:30:21.912880 8712 raft_consensus.cc:3037] T fdb20d7f3ada476096f320db6d088346 P b8e3bd4d570d422d8dd77fd3c581232b [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:30:21.912879 8489 raft_consensus.cc:3037] T fdb20d7f3ada476096f320db6d088346 P 0b83ec152b074f80af2848108d3af254 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:30:21.916371 8712 raft_consensus.cc:2454] T fdb20d7f3ada476096f320db6d088346 P b8e3bd4d570d422d8dd77fd3c581232b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c0f7f24b8f0420a93c04bdf627b58e1 in term 1.
I20260811 06:30:21.916371 8489 raft_consensus.cc:2454] T fdb20d7f3ada476096f320db6d088346 P 0b83ec152b074f80af2848108d3af254 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c0f7f24b8f0420a93c04bdf627b58e1 in term 1.
I20260811 06:30:21.916971 8827 leader_election.cc:302] T fdb20d7f3ada476096f320db6d088346 P 7c0f7f24b8f0420a93c04bdf627b58e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0b83ec152b074f80af2848108d3af254, 7c0f7f24b8f0420a93c04bdf627b58e1, b8e3bd4d570d422d8dd77fd3c581232b; no voters:
I20260811 06:30:21.917407 9059 raft_consensus.cc:2781] T fdb20d7f3ada476096f320db6d088346 P 7c0f7f24b8f0420a93c04bdf627b58e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:30:21.918043 9059 raft_consensus.cc:686] T fdb20d7f3ada476096f320db6d088346 P 7c0f7f24b8f0420a93c04bdf627b58e1 [term 1 LEADER]: Becoming Leader. State: Replica: 7c0f7f24b8f0420a93c04bdf627b58e1, State: Running, Role: LEADER
I20260811 06:30:21.918853 9059 consensus_queue.cc:237] T fdb20d7f3ada476096f320db6d088346 P 7c0f7f24b8f0420a93c04bdf627b58e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b83ec152b074f80af2848108d3af254" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 40357 } } peers { permanent_uuid: "b8e3bd4d570d422d8dd77fd3c581232b" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 45127 } } peers { permanent_uuid: "7c0f7f24b8f0420a93c04bdf627b58e1" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 44989 } }
I20260811 06:30:21.924682 7047 catalog_manager.cc:5495] T fdb20d7f3ada476096f320db6d088346 P 7c0f7f24b8f0420a93c04bdf627b58e1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c0f7f24b8f0420a93c04bdf627b58e1 (127.28.239.195). New cstate: current_term: 1 leader_uuid: "7c0f7f24b8f0420a93c04bdf627b58e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b83ec152b074f80af2848108d3af254" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 40357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8e3bd4d570d422d8dd77fd3c581232b" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 45127 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c0f7f24b8f0420a93c04bdf627b58e1" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 44989 } health_report { overall_health: HEALTHY } } }
I20260811 06:30:21.963169 9048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:21.957163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43386 (local address 127.28.239.195:44989)
0811 06:30:21.957371 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:21.957376 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:21.957668 (+ 292us) server_negotiation.cc:407] Connection header received
0811 06:30:21.957890 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:21.957894 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:21.957987 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:21.958149 (+ 162us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:30:21.958536 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:21.959339 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:21.960743 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:21.961257 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:21.961376 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:21.961970 (+ 594us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:30:21.962594 (+ 624us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:30:21.962721 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:21.962858 (+ 137us) server_negotiation.cc:299] Negotiation successful
0811 06:30:21.962937 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 06:30:21.972306 8489 raft_consensus.cc:1261] T fdb20d7f3ada476096f320db6d088346 P 0b83ec152b074f80af2848108d3af254 [term 1 FOLLOWER]: Refusing update from remote peer 7c0f7f24b8f0420a93c04bdf627b58e1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:30:21.972306 8712 raft_consensus.cc:1261] T fdb20d7f3ada476096f320db6d088346 P b8e3bd4d570d422d8dd77fd3c581232b [term 1 FOLLOWER]: Refusing update from remote peer 7c0f7f24b8f0420a93c04bdf627b58e1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:30:21.973511 9059 consensus_queue.cc:1035] T fdb20d7f3ada476096f320db6d088346 P 7c0f7f24b8f0420a93c04bdf627b58e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e3bd4d570d422d8dd77fd3c581232b" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 45127 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:30:21.973968 9061 consensus_queue.cc:1035] T fdb20d7f3ada476096f320db6d088346 P 7c0f7f24b8f0420a93c04bdf627b58e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b83ec152b074f80af2848108d3af254" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 40357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:30:22.018460 7047 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54094:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:30:22.018774 7047 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54094:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:30:22.029667 7047 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c1478483ef29416fb130a1d621aafbaa: Sending DeleteTablet for 3 replicas of tablet fdb20d7f3ada476096f320db6d088346
I20260811 06:30:22.031109 8460 tablet_service.cc:1514] Processing DeleteTablet for tablet fdb20d7f3ada476096f320db6d088346 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:30:22 UTC) from {username='oryx', principal='oryx/127.28.239.254@KRBTEST.COM'} at 127.0.0.1:40582
I20260811 06:30:22.031174 8678 tablet_service.cc:1514] Processing DeleteTablet for tablet fdb20d7f3ada476096f320db6d088346 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:30:22 UTC) from {username='oryx', principal='oryx/127.28.239.254@KRBTEST.COM'} at 127.0.0.1:33712
I20260811 06:30:22.039062 8896 tablet_service.cc:1514] Processing DeleteTablet for tablet fdb20d7f3ada476096f320db6d088346 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:30:22 UTC) from {username='oryx', principal='oryx/127.28.239.254@KRBTEST.COM'} at 127.0.0.1:43374
I20260811 06:30:22.046875 9083 tablet_replica.cc:330] T fdb20d7f3ada476096f320db6d088346 P 0b83ec152b074f80af2848108d3af254: stopping tablet replica
I20260811 06:30:22.047422 9083 raft_consensus.cc:2229] T fdb20d7f3ada476096f320db6d088346 P 0b83ec152b074f80af2848108d3af254 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:30:22.047950 9083 raft_consensus.cc:2258] T fdb20d7f3ada476096f320db6d088346 P 0b83ec152b074f80af2848108d3af254 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:30:22.048683 9084 tablet_replica.cc:330] T fdb20d7f3ada476096f320db6d088346 P b8e3bd4d570d422d8dd77fd3c581232b: stopping tablet replica
I20260811 06:30:22.049111 9084 raft_consensus.cc:2229] T fdb20d7f3ada476096f320db6d088346 P b8e3bd4d570d422d8dd77fd3c581232b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:30:22.049588 9084 raft_consensus.cc:2258] T fdb20d7f3ada476096f320db6d088346 P b8e3bd4d570d422d8dd77fd3c581232b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:30:22.050377 9086 tablet_replica.cc:330] T fdb20d7f3ada476096f320db6d088346 P 7c0f7f24b8f0420a93c04bdf627b58e1: stopping tablet replica
I20260811 06:30:22.050792 9086 raft_consensus.cc:2229] T fdb20d7f3ada476096f320db6d088346 P 7c0f7f24b8f0420a93c04bdf627b58e1 [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:30:22.051911 9084 ts_tablet_manager.cc:1905] T fdb20d7f3ada476096f320db6d088346 P b8e3bd4d570d422d8dd77fd3c581232b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:30:22.052349 9086 raft_consensus.cc:2258] T fdb20d7f3ada476096f320db6d088346 P 7c0f7f24b8f0420a93c04bdf627b58e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:30:22.054657 9086 ts_tablet_manager.cc:1905] T fdb20d7f3ada476096f320db6d088346 P 7c0f7f24b8f0420a93c04bdf627b58e1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:30:22.055568 9083 ts_tablet_manager.cc:1905] T fdb20d7f3ada476096f320db6d088346 P 0b83ec152b074f80af2848108d3af254: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:30:22.064013 9083 ts_tablet_manager.cc:1918] T fdb20d7f3ada476096f320db6d088346 P 0b83ec152b074f80af2848108d3af254: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:30:22.064278 9083 log.cc:1196] T fdb20d7f3ada476096f320db6d088346 P 0b83ec152b074f80af2848108d3af254: Deleting WAL directory at /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-0/wal/wals/fdb20d7f3ada476096f320db6d088346
I20260811 06:30:22.064512 9086 ts_tablet_manager.cc:1918] T fdb20d7f3ada476096f320db6d088346 P 7c0f7f24b8f0420a93c04bdf627b58e1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:30:22.064719 9086 log.cc:1196] T fdb20d7f3ada476096f320db6d088346 P 7c0f7f24b8f0420a93c04bdf627b58e1: Deleting WAL directory at /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-2/wal/wals/fdb20d7f3ada476096f320db6d088346
I20260811 06:30:22.064916 9083 ts_tablet_manager.cc:1939] T fdb20d7f3ada476096f320db6d088346 P 0b83ec152b074f80af2848108d3af254: Deleting consensus metadata
I20260811 06:30:22.065399 9086 ts_tablet_manager.cc:1939] T fdb20d7f3ada476096f320db6d088346 P 7c0f7f24b8f0420a93c04bdf627b58e1: Deleting consensus metadata
I20260811 06:30:22.067812 7029 catalog_manager.cc:4841] TS 7c0f7f24b8f0420a93c04bdf627b58e1 (127.28.239.195:44989): tablet fdb20d7f3ada476096f320db6d088346 (table test-table [id=1a8119a454df42f29ad7b77c9d7d66bd]) successfully deleted
I20260811 06:30:22.069808 7026 catalog_manager.cc:4841] TS 0b83ec152b074f80af2848108d3af254 (127.28.239.193:40357): tablet fdb20d7f3ada476096f320db6d088346 (table test-table [id=1a8119a454df42f29ad7b77c9d7d66bd]) successfully deleted
I20260811 06:30:22.072710 9084 ts_tablet_manager.cc:1918] T fdb20d7f3ada476096f320db6d088346 P b8e3bd4d570d422d8dd77fd3c581232b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:30:22.072898 9084 log.cc:1196] T fdb20d7f3ada476096f320db6d088346 P b8e3bd4d570d422d8dd77fd3c581232b: Deleting WAL directory at /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-1/wal/wals/fdb20d7f3ada476096f320db6d088346
I20260811 06:30:22.073469 9084 ts_tablet_manager.cc:1939] T fdb20d7f3ada476096f320db6d088346 P b8e3bd4d570d422d8dd77fd3c581232b: Deleting consensus metadata
I20260811 06:30:22.075314 7028 catalog_manager.cc:4841] TS b8e3bd4d570d422d8dd77fd3c581232b (127.28.239.194:45127): tablet fdb20d7f3ada476096f320db6d088346 (table test-table [id=1a8119a454df42f29ad7b77c9d7d66bd]) successfully deleted
I20260811 06:30:22.075930 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 7004
I20260811 06:30:22.077245 7101 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:30:22.107301 9089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:22.050884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55938 (local address 127.28.239.253:45951)
0811 06:30:22.082495 (+ 31611us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:22.082500 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:22.082521 (+ 21us) server_negotiation.cc:407] Connection header received
0811 06:30:22.082579 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:22.082583 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:22.082649 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:22.082819 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:22.085505 (+ 2686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:22.086730 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:22.088766 (+ 2036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:22.089465 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:22.089656 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:22.099488 (+ 9832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:22.099497 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:22.099503 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:30:22.099557 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:22.103775 (+ 4218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:22.104828 (+ 1053us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:22.104835 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:22.104839 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:22.105020 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:22.105520 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:22.105542 (+ 22us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:22.105547 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:22.105968 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:22.106263 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:22.106653 (+ 390us) server_negotiation.cc:299] Negotiation successful
0811 06:30:22.106904 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31488,"thread_start_us":31348,"threads_started":1}
I20260811 06:30:22.124822 9091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:22.065951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45242 (local address 127.28.239.252:43777)
0811 06:30:22.083486 (+ 17535us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:22.083494 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:22.107570 (+ 24076us) server_negotiation.cc:407] Connection header received
0811 06:30:22.107767 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:22.107773 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:22.107861 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:22.107985 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:22.109809 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:22.111181 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:22.113117 (+ 1936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:22.113860 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:22.113993 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:22.117468 (+ 3475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:22.117476 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:22.117481 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:30:22.117556 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:22.121576 (+ 4020us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:22.122387 (+ 811us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:22.122392 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:22.122396 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:22.122564 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:22.122988 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:22.122993 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:22.122996 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:22.123376 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:22.123634 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:22.123983 (+ 349us) server_negotiation.cc:299] Negotiation successful
0811 06:30:22.124223 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17364,"thread_start_us":17536,"threads_started":1}
I20260811 06:30:22.213651 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 7004
I20260811 06:30:22.266083 9089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:22.249100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55950 (local address 127.28.239.253:45951)
0811 06:30:22.249368 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:22.249377 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:22.250108 (+ 731us) server_negotiation.cc:407] Connection header received
0811 06:30:22.250188 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:22.250194 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:22.250294 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:22.250389 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:22.253014 (+ 2625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:22.254003 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:22.255675 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:22.256417 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:22.256561 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:22.259825 (+ 3264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:22.259832 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:22.259837 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:30:22.259880 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:22.263100 (+ 3220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:22.263863 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:22.263869 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:22.263873 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:22.264025 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:22.264412 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:22.264417 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:22.264421 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:22.264813 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:22.265093 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:22.265458 (+ 365us) server_negotiation.cc:299] Negotiation successful
0811 06:30:22.265726 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260811 06:30:22.269106 9091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:22.250556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45248 (local address 127.28.239.252:43777)
0811 06:30:22.250806 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:22.250813 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:22.251287 (+ 474us) server_negotiation.cc:407] Connection header received
0811 06:30:22.251506 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:22.251512 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:22.251585 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:22.251697 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:22.253620 (+ 1923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:22.254609 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:22.257785 (+ 3176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:22.258468 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:22.258617 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:22.262520 (+ 3903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:22.262526 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:22.262533 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:30:22.262575 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:22.266290 (+ 3715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:22.267120 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:22.267125 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:22.267128 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:22.267294 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:22.267759 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:22.267762 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:22.267765 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:22.268036 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:22.268244 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:22.268638 (+ 394us) server_negotiation.cc:299] Negotiation successful
0811 06:30:22.268816 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260811 06:30:23.016429 9103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:23.015579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:40129 (local address 127.28.239.195:51283)
0811 06:30:23.016137 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:23.016262 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:40129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":245,"threads_started":1}
W20260811 06:30:23.017091 8999 heartbeater.cc:644] Failed to heartbeat to 127.28.239.254:40129 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.28.239.254:40129: connect: Connection refused (error 111)
I20260811 06:30:23.037426 9105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:23.036662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:40129 (local address 127.28.239.193:59387)
0811 06:30:23.037131 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:23.037251 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:40129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":224,"threads_started":1}
W20260811 06:30:23.037864 8566 heartbeater.cc:644] Failed to heartbeat to 127.28.239.254:40129 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.28.239.254:40129: connect: Connection refused (error 111)
I20260811 06:30:23.049777 9107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:23.048877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:40129 (local address 127.28.239.194:57891)
0811 06:30:23.049328 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:23.049480 (+ 152us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:40129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":223,"threads_started":1}
W20260811 06:30:23.050318 8782 heartbeater.cc:644] Failed to heartbeat to 127.28.239.254:40129 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.28.239.254:40129: connect: Connection refused (error 111)
I20260811 06:30:23.529410 9108 raft_consensus.cc:482] T 00000000000000000000000000000000 P 81d907d1c7ce487da25ae51b57e094e3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c1478483ef29416fb130a1d621aafbaa)
I20260811 06:30:23.529812 9108 raft_consensus.cc:504] T 00000000000000000000000000000000 P 81d907d1c7ce487da25ae51b57e094e3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c1478483ef29416fb130a1d621aafbaa" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 40129 } } peers { permanent_uuid: "81d907d1c7ce487da25ae51b57e094e3" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 45951 } attrs { promote: false } } peers { permanent_uuid: "3dbd88a041b4459980dab85a3d3bfaf7" member_type: VOTER last_known_addr { host: "127.28.239.252" port: 43777 } attrs { promote: false } }
I20260811 06:30:23.531337 9108 leader_election.cc:288] T 00000000000000000000000000000000 P 81d907d1c7ce487da25ae51b57e094e3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c1478483ef29416fb130a1d621aafbaa (127.28.239.254:40129), 3dbd88a041b4459980dab85a3d3bfaf7 (127.28.239.252:43777)
I20260811 06:30:23.531385 9109 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3dbd88a041b4459980dab85a3d3bfaf7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c1478483ef29416fb130a1d621aafbaa)
I20260811 06:30:23.531690 9109 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3dbd88a041b4459980dab85a3d3bfaf7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c1478483ef29416fb130a1d621aafbaa" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 40129 } } peers { permanent_uuid: "81d907d1c7ce487da25ae51b57e094e3" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 45951 } attrs { promote: false } } peers { permanent_uuid: "3dbd88a041b4459980dab85a3d3bfaf7" member_type: VOTER last_known_addr { host: "127.28.239.252" port: 43777 } attrs { promote: false } }
I20260811 06:30:23.532877 9112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:23.531461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:40129 (local address 127.0.0.1:54502)
0811 06:30:23.532335 (+ 874us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:23.532505 (+ 170us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:40129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":744,"thread_start_us":601,"threads_started":1}
I20260811 06:30:23.534258 9109 leader_election.cc:288] T 00000000000000000000000000000000 P 3dbd88a041b4459980dab85a3d3bfaf7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c1478483ef29416fb130a1d621aafbaa (127.28.239.254:40129), 81d907d1c7ce487da25ae51b57e094e3 (127.28.239.253:45951)
I20260811 06:30:23.535506 9115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:23.533346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:40129 (local address 127.0.0.1:54504)
0811 06:30:23.534867 (+ 1521us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:23.535123 (+ 256us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:40129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1388,"thread_start_us":1220,"threads_started":1}
W20260811 06:30:23.535953 8262 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.28.239.254:40129: connect: Connection refused (error 111)
W20260811 06:30:23.534132 7969 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.28.239.254:40129: connect: Connection refused (error 111)
I20260811 06:30:23.542630 9115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:23.542067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:40129 (local address 127.0.0.1:54512)
0811 06:30:23.542298 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:23.542415 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:40129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129}
W20260811 06:30:23.542894 8262 leader_election.cc:334] T 00000000000000000000000000000000 P 3dbd88a041b4459980dab85a3d3bfaf7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c1478483ef29416fb130a1d621aafbaa (127.28.239.254:40129): Network error: Client connection negotiation failed: client connection to 127.28.239.254:40129: connect: Connection refused (error 111)
I20260811 06:30:23.545059 9113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:23.544307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:40129 (local address 127.0.0.1:54528)
0811 06:30:23.544476 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:23.544774 (+ 298us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:40129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260811 06:30:23.545318 7969 leader_election.cc:334] T 00000000000000000000000000000000 P 81d907d1c7ce487da25ae51b57e094e3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c1478483ef29416fb130a1d621aafbaa (127.28.239.254:40129): Network error: Client connection negotiation failed: client connection to 127.28.239.254:40129: connect: Connection refused (error 111)
I20260811 06:30:23.545483 9114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:23.532745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45264 (local address 127.28.239.252:43777)
0811 06:30:23.536428 (+ 3683us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:23.536438 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:23.536464 (+ 26us) server_negotiation.cc:407] Connection header received
0811 06:30:23.536549 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:23.536557 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:23.536658 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:23.537523 (+ 865us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:30:23.537821 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:23.539547 (+ 1726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:23.542950 (+ 3403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:23.544236 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:23.544408 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:23.544488 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:23.544869 (+ 381us) server_negotiation.cc:299] Negotiation successful
0811 06:30:23.545016 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3371,"thread_start_us":3216,"threads_started":1}
I20260811 06:30:23.545415 9112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:23.532472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.252:43777 (local address 127.0.0.1:45264)
0811 06:30:23.533283 (+ 811us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:23.533307 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:23.533485 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:23.536868 (+ 3383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:23.536877 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:23.536950 (+ 73us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:30:23.537345 (+ 395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:23.537362 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:23.539756 (+ 2394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:23.539765 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:23.542570 (+ 2805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:23.542592 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:23.544391 (+ 1799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:23.544397 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:23.544623 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:23.544678 (+ 55us) client_negotiation.cc:769] Sending connection context
0811 06:30:23.544815 (+ 137us) client_negotiation.cc:240] Negotiation successful
0811 06:30:23.544973 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":665,"thread_start_us":1340,"threads_started":1}
I20260811 06:30:23.546167 8304 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "81d907d1c7ce487da25ae51b57e094e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "3dbd88a041b4459980dab85a3d3bfaf7" is_pre_election: true
I20260811 06:30:23.546527 8304 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3dbd88a041b4459980dab85a3d3bfaf7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 81d907d1c7ce487da25ae51b57e094e3 in term 1.
I20260811 06:30:23.547060 7969 leader_election.cc:302] T 00000000000000000000000000000000 P 81d907d1c7ce487da25ae51b57e094e3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3dbd88a041b4459980dab85a3d3bfaf7, 81d907d1c7ce487da25ae51b57e094e3; no voters: c1478483ef29416fb130a1d621aafbaa
I20260811 06:30:23.547472 9108 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 81d907d1c7ce487da25ae51b57e094e3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 06:30:23.547633 9108 raft_consensus.cc:482] T 00000000000000000000000000000000 P 81d907d1c7ce487da25ae51b57e094e3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader c1478483ef29416fb130a1d621aafbaa)
I20260811 06:30:23.547803 9108 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 81d907d1c7ce487da25ae51b57e094e3 [term 1 FOLLOWER]: Advancing to term 2
I20260811 06:30:23.549556 9117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:23.534435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.253:45951 (local address 127.0.0.1:55956)
0811 06:30:23.535262 (+ 827us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:23.535286 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:23.535454 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:23.542234 (+ 6780us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:23.542239 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:23.542267 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:30:23.542678 (+ 411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:23.542691 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:23.544764 (+ 2073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:23.544770 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:23.547083 (+ 2313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:23.547096 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:23.548686 (+ 1590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:23.548692 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:23.548903 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:23.548938 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 06:30:23.549042 (+ 104us) client_negotiation.cc:240] Negotiation successful
0811 06:30:23.549192 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":629,"thread_start_us":489,"threads_started":1}
I20260811 06:30:23.549511 9118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:23.534787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55956 (local address 127.28.239.253:45951)
0811 06:30:23.541891 (+ 7104us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:23.541900 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:23.541920 (+ 20us) server_negotiation.cc:407] Connection header received
0811 06:30:23.541983 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:23.541988 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:23.542061 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:23.542855 (+ 794us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:30:23.543170 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:23.544533 (+ 1363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:23.547248 (+ 2715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:23.548488 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:23.548635 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:23.548687 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:23.549093 (+ 406us) server_negotiation.cc:299] Negotiation successful
0811 06:30:23.549227 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6965,"thread_start_us":6753,"threads_started":1}
I20260811 06:30:23.550262 8012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3dbd88a041b4459980dab85a3d3bfaf7" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "81d907d1c7ce487da25ae51b57e094e3" is_pre_election: true
I20260811 06:30:23.551167 9108 raft_consensus.cc:504] T 00000000000000000000000000000000 P 81d907d1c7ce487da25ae51b57e094e3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c1478483ef29416fb130a1d621aafbaa" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 40129 } } peers { permanent_uuid: "81d907d1c7ce487da25ae51b57e094e3" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 45951 } attrs { promote: false } } peers { permanent_uuid: "3dbd88a041b4459980dab85a3d3bfaf7" member_type: VOTER last_known_addr { host: "127.28.239.252" port: 43777 } attrs { promote: false } }
I20260811 06:30:23.551604 8012 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 81d907d1c7ce487da25ae51b57e094e3 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3dbd88a041b4459980dab85a3d3bfaf7 in current term 2: Already voted for candidate 81d907d1c7ce487da25ae51b57e094e3 in this term.
I20260811 06:30:23.551981 9108 leader_election.cc:288] T 00000000000000000000000000000000 P 81d907d1c7ce487da25ae51b57e094e3 [CANDIDATE]: Term 2 election: Requested vote from peers c1478483ef29416fb130a1d621aafbaa (127.28.239.254:40129), 3dbd88a041b4459980dab85a3d3bfaf7 (127.28.239.252:43777)
I20260811 06:30:23.552449 9112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:23.552028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:40129 (local address 127.0.0.1:54542)
0811 06:30:23.552245 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:23.552326 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:40129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
I20260811 06:30:23.552693 8265 leader_election.cc:302] T 00000000000000000000000000000000 P 3dbd88a041b4459980dab85a3d3bfaf7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3dbd88a041b4459980dab85a3d3bfaf7; no voters: 81d907d1c7ce487da25ae51b57e094e3, c1478483ef29416fb130a1d621aafbaa
I20260811 06:30:23.553105 9109 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3dbd88a041b4459980dab85a3d3bfaf7 [term 1 FOLLOWER]: Advancing to term 2
I20260811 06:30:23.553188 8304 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "81d907d1c7ce487da25ae51b57e094e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "3dbd88a041b4459980dab85a3d3bfaf7"
I20260811 06:30:23.553818 9112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:23.553284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:40129 (local address 127.0.0.1:54558)
0811 06:30:23.553579 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:23.553715 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:40129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173}
W20260811 06:30:23.553968 7969 leader_election.cc:334] T 00000000000000000000000000000000 P 81d907d1c7ce487da25ae51b57e094e3 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer c1478483ef29416fb130a1d621aafbaa (127.28.239.254:40129): Network error: Client connection negotiation failed: client connection to 127.28.239.254:40129: connect: Connection refused (error 111)
I20260811 06:30:23.555459 9109 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 3dbd88a041b4459980dab85a3d3bfaf7 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 06:30:23.557796 8304 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3dbd88a041b4459980dab85a3d3bfaf7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 81d907d1c7ce487da25ae51b57e094e3 in term 2.
I20260811 06:30:23.558566 7969 leader_election.cc:302] T 00000000000000000000000000000000 P 81d907d1c7ce487da25ae51b57e094e3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3dbd88a041b4459980dab85a3d3bfaf7, 81d907d1c7ce487da25ae51b57e094e3; no voters: c1478483ef29416fb130a1d621aafbaa
I20260811 06:30:23.558975 9108 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 81d907d1c7ce487da25ae51b57e094e3 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 06:30:23.559446 9108 raft_consensus.cc:686] T 00000000000000000000000000000000 P 81d907d1c7ce487da25ae51b57e094e3 [term 2 LEADER]: Becoming Leader. State: Replica: 81d907d1c7ce487da25ae51b57e094e3, State: Running, Role: LEADER
I20260811 06:30:23.559865 9108 consensus_queue.cc:237] T 00000000000000000000000000000000 P 81d907d1c7ce487da25ae51b57e094e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c1478483ef29416fb130a1d621aafbaa" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 40129 } } peers { permanent_uuid: "81d907d1c7ce487da25ae51b57e094e3" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 45951 } attrs { promote: false } } peers { permanent_uuid: "3dbd88a041b4459980dab85a3d3bfaf7" member_type: VOTER last_known_addr { host: "127.28.239.252" port: 43777 } attrs { promote: false } }
I20260811 06:30:23.562009 9123 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81d907d1c7ce487da25ae51b57e094e3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 81d907d1c7ce487da25ae51b57e094e3. Latest consensus state: current_term: 2 leader_uuid: "81d907d1c7ce487da25ae51b57e094e3" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c1478483ef29416fb130a1d621aafbaa" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 40129 } } peers { permanent_uuid: "81d907d1c7ce487da25ae51b57e094e3" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 45951 } attrs { promote: false } } peers { permanent_uuid: "3dbd88a041b4459980dab85a3d3bfaf7" member_type: VOTER last_known_addr { host: "127.28.239.252" port: 43777 } attrs { promote: false } } }
I20260811 06:30:23.562498 9123 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81d907d1c7ce487da25ae51b57e094e3 [sys.catalog]: This master's current role is: LEADER
I20260811 06:30:23.563542 9125 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:30:23.568070 9125 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:30:23.568992 9125 catalog_manager.cc:1259] Loaded cluster ID: e6f352fda9a24973ae7bc5beafb7d908
I20260811 06:30:23.569119 9125 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:30:23.570479 9125 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:30:23.571403 9125 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 81d907d1c7ce487da25ae51b57e094e3: Loaded TSK: 0
I20260811 06:30:23.572407 9125 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:30:23.619760 7992 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55950:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:30:23.627349 7992 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:30:23.631839 9112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:23.631397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:40129 (local address 127.0.0.1:54588)
0811 06:30:23.631604 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:23.631694 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:40129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260811 06:30:23.631850 8304 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3dbd88a041b4459980dab85a3d3bfaf7 [term 2 FOLLOWER]: Refusing update from remote peer 81d907d1c7ce487da25ae51b57e094e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260811 06:30:23.632710 9123 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 81d907d1c7ce487da25ae51b57e094e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3dbd88a041b4459980dab85a3d3bfaf7" member_type: VOTER last_known_addr { host: "127.28.239.252" port: 43777 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260811 06:30:23.633353 9112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:23.632872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:40129 (local address 127.0.0.1:54600)
0811 06:30:23.633115 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:23.633198 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:40129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
W20260811 06:30:23.633562 7969 consensus_peers.cc:487] T 00000000000000000000000000000000 P 81d907d1c7ce487da25ae51b57e094e3 -> Peer c1478483ef29416fb130a1d621aafbaa (127.28.239.254:40129): Couldn't send request to peer c1478483ef29416fb130a1d621aafbaa. Status: Network error: Client connection negotiation failed: client connection to 127.28.239.254:40129: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 06:30:23.636183 9109 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3dbd88a041b4459980dab85a3d3bfaf7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 81d907d1c7ce487da25ae51b57e094e3. Latest consensus state: current_term: 2 leader_uuid: "81d907d1c7ce487da25ae51b57e094e3" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c1478483ef29416fb130a1d621aafbaa" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 40129 } } peers { permanent_uuid: "81d907d1c7ce487da25ae51b57e094e3" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 45951 } attrs { promote: false } } peers { permanent_uuid: "3dbd88a041b4459980dab85a3d3bfaf7" member_type: VOTER last_known_addr { host: "127.28.239.252" port: 43777 } attrs { promote: false } } }
I20260811 06:30:23.636633 9109 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3dbd88a041b4459980dab85a3d3bfaf7 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:30:23.638290 9123 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81d907d1c7ce487da25ae51b57e094e3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "81d907d1c7ce487da25ae51b57e094e3" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c1478483ef29416fb130a1d621aafbaa" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 40129 } } peers { permanent_uuid: "81d907d1c7ce487da25ae51b57e094e3" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 45951 } attrs { promote: false } } peers { permanent_uuid: "3dbd88a041b4459980dab85a3d3bfaf7" member_type: VOTER last_known_addr { host: "127.28.239.252" port: 43777 } attrs { promote: false } } }
I20260811 06:30:23.638617 9123 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81d907d1c7ce487da25ae51b57e094e3 [sys.catalog]: This master's current role is: LEADER
I20260811 06:30:23.639024 9109 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3dbd88a041b4459980dab85a3d3bfaf7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "81d907d1c7ce487da25ae51b57e094e3" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c1478483ef29416fb130a1d621aafbaa" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 40129 } } peers { permanent_uuid: "81d907d1c7ce487da25ae51b57e094e3" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 45951 } attrs { promote: false } } peers { permanent_uuid: "3dbd88a041b4459980dab85a3d3bfaf7" member_type: VOTER last_known_addr { host: "127.28.239.252" port: 43777 } attrs { promote: false } } }
I20260811 06:30:23.639357 9109 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3dbd88a041b4459980dab85a3d3bfaf7 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:30:23.658984 9135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:23.647043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43388 (local address 127.28.239.195:44989)
0811 06:30:23.651155 (+ 4112us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:23.651161 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:23.651181 (+ 20us) server_negotiation.cc:407] Connection header received
0811 06:30:23.651254 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:23.651258 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:23.651317 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:23.651459 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:30:23.651861 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:23.652981 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:23.656323 (+ 3342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:23.657694 (+ 1371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:23.658304 (+ 610us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:23.658411 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:23.658467 (+ 56us) server_negotiation.cc:299] Negotiation successful
0811 06:30:23.658634 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3986,"thread_start_us":3876,"threads_started":1}
I20260811 06:30:23.659003 9112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:23.646960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.195:44989 (local address 127.0.0.1:43388)
0811 06:30:23.647226 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:23.647247 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:23.647355 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:23.651478 (+ 4123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:23.651483 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:23.651510 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:30:23.651688 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:23.651698 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:23.653203 (+ 1505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:23.653209 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:23.654723 (+ 1514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:23.654737 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:23.657931 (+ 3194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:23.657940 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:23.658152 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:23.658265 (+ 113us) client_negotiation.cc:769] Sending connection context
0811 06:30:23.658381 (+ 116us) client_negotiation.cc:240] Negotiation successful
0811 06:30:23.658629 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"mutex_wait_us":131}
I20260811 06:30:23.659016 9113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:23.647566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.193:40357 (local address 127.0.0.1:40584)
0811 06:30:23.647740 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:23.647764 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:23.647888 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:23.650574 (+ 2686us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:23.650578 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:23.650608 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:30:23.650907 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:23.650916 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:23.653155 (+ 2239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:23.653163 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:23.656046 (+ 2883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:23.656062 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:23.658044 (+ 1982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:23.658053 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:23.658306 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:23.658353 (+ 47us) client_negotiation.cc:769] Sending connection context
0811 06:30:23.658451 (+ 98us) client_negotiation.cc:240] Negotiation successful
0811 06:30:23.658640 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66,"mutex_wait_us":1386}
I20260811 06:30:23.659705 9140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:23.648054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40584 (local address 127.28.239.193:40357)
0811 06:30:23.650147 (+ 2093us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:23.650157 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:23.650186 (+ 29us) server_negotiation.cc:407] Connection header received
0811 06:30:23.650293 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:23.650302 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:23.650418 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:23.650563 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:30:23.651085 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:23.652941 (+ 1856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:23.656251 (+ 3310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:23.657718 (+ 1467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:23.659114 (+ 1396us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:23.659195 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:23.659251 (+ 56us) server_negotiation.cc:299] Negotiation successful
0811 06:30:23.659387 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1910,"thread_start_us":1788,"threads_started":1}
I20260811 06:30:23.660723 8460 tablet_service.cc:1467] Processing CreateTablet for tablet 953060620da94a6381aba2a93be8e9cb (DEFAULT_TABLE table=test-table [id=a7b99412561a479dae90bad34a2cb5c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:30:23.661211 8460 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 953060620da94a6381aba2a93be8e9cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:30:23.662084 8896 tablet_service.cc:1467] Processing CreateTablet for tablet 953060620da94a6381aba2a93be8e9cb (DEFAULT_TABLE table=test-table [id=a7b99412561a479dae90bad34a2cb5c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:30:23.662549 8896 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 953060620da94a6381aba2a93be8e9cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:30:23.667636 9141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:23.648418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33718 (local address 127.28.239.194:45127)
0811 06:30:23.657462 (+ 9044us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:23.657469 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:23.657491 (+ 22us) server_negotiation.cc:407] Connection header received
0811 06:30:23.657601 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:23.657607 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:23.657697 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:23.657819 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:30:23.658624 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:23.659699 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:23.665468 (+ 5769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:23.666485 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:23.666723 (+ 238us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:23.666815 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:23.667136 (+ 321us) server_negotiation.cc:299] Negotiation successful
0811 06:30:23.667287 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8901,"thread_start_us":8784,"threads_started":1}
I20260811 06:30:23.667718 9138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:23.647899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.194:45127 (local address 127.0.0.1:33718)
0811 06:30:23.648545 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:23.648565 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:23.649001 (+ 436us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:23.657865 (+ 8864us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:23.657870 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:23.657904 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:30:23.658413 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:23.658428 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:23.659890 (+ 1462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:23.659897 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:23.664785 (+ 4888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:23.664803 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:23.666757 (+ 1954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:23.666765 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:23.666942 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:23.666970 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 06:30:23.667097 (+ 127us) client_negotiation.cc:240] Negotiation successful
0811 06:30:23.667257 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":527,"mutex_wait_us":6,"thread_start_us":294,"threads_started":1}
I20260811 06:30:23.668835 9142 tablet_bootstrap.cc:492] T 953060620da94a6381aba2a93be8e9cb P 0b83ec152b074f80af2848108d3af254: Bootstrap starting.
I20260811 06:30:23.668819 8678 tablet_service.cc:1467] Processing CreateTablet for tablet 953060620da94a6381aba2a93be8e9cb (DEFAULT_TABLE table=test-table [id=a7b99412561a479dae90bad34a2cb5c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:30:23.669320 8678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 953060620da94a6381aba2a93be8e9cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:30:23.669579 9144 tablet_bootstrap.cc:492] T 953060620da94a6381aba2a93be8e9cb P 7c0f7f24b8f0420a93c04bdf627b58e1: Bootstrap starting.
I20260811 06:30:23.671330 9144 tablet_bootstrap.cc:654] T 953060620da94a6381aba2a93be8e9cb P 7c0f7f24b8f0420a93c04bdf627b58e1: Neither blocks nor log segments found. Creating new log.
I20260811 06:30:23.671553 9142 tablet_bootstrap.cc:654] T 953060620da94a6381aba2a93be8e9cb P 0b83ec152b074f80af2848108d3af254: Neither blocks nor log segments found. Creating new log.
I20260811 06:30:23.673753 9144 tablet_bootstrap.cc:492] T 953060620da94a6381aba2a93be8e9cb P 7c0f7f24b8f0420a93c04bdf627b58e1: No bootstrap required, opened a new log
I20260811 06:30:23.673837 9142 tablet_bootstrap.cc:492] T 953060620da94a6381aba2a93be8e9cb P 0b83ec152b074f80af2848108d3af254: No bootstrap required, opened a new log
I20260811 06:30:23.673920 9144 ts_tablet_manager.cc:1397] T 953060620da94a6381aba2a93be8e9cb P 7c0f7f24b8f0420a93c04bdf627b58e1: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.003s
I20260811 06:30:23.673983 9142 ts_tablet_manager.cc:1397] T 953060620da94a6381aba2a93be8e9cb P 0b83ec152b074f80af2848108d3af254: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 06:30:23.674674 9142 raft_consensus.cc:348] T 953060620da94a6381aba2a93be8e9cb P 0b83ec152b074f80af2848108d3af254 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f7f24b8f0420a93c04bdf627b58e1" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 44989 } } peers { permanent_uuid: "0b83ec152b074f80af2848108d3af254" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 40357 } } peers { permanent_uuid: "b8e3bd4d570d422d8dd77fd3c581232b" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 45127 } }
I20260811 06:30:23.674894 9142 raft_consensus.cc:374] T 953060620da94a6381aba2a93be8e9cb P 0b83ec152b074f80af2848108d3af254 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:30:23.674986 9142 raft_consensus.cc:729] T 953060620da94a6381aba2a93be8e9cb P 0b83ec152b074f80af2848108d3af254 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b83ec152b074f80af2848108d3af254, State: Initialized, Role: FOLLOWER
I20260811 06:30:23.674814 9144 raft_consensus.cc:348] T 953060620da94a6381aba2a93be8e9cb P 7c0f7f24b8f0420a93c04bdf627b58e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f7f24b8f0420a93c04bdf627b58e1" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 44989 } } peers { permanent_uuid: "0b83ec152b074f80af2848108d3af254" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 40357 } } peers { permanent_uuid: "b8e3bd4d570d422d8dd77fd3c581232b" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 45127 } }
I20260811 06:30:23.675122 9144 raft_consensus.cc:374] T 953060620da94a6381aba2a93be8e9cb P 7c0f7f24b8f0420a93c04bdf627b58e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:30:23.675249 9144 raft_consensus.cc:729] T 953060620da94a6381aba2a93be8e9cb P 7c0f7f24b8f0420a93c04bdf627b58e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c0f7f24b8f0420a93c04bdf627b58e1, State: Initialized, Role: FOLLOWER
I20260811 06:30:23.675335 9142 consensus_queue.cc:260] T 953060620da94a6381aba2a93be8e9cb P 0b83ec152b074f80af2848108d3af254 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f7f24b8f0420a93c04bdf627b58e1" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 44989 } } peers { permanent_uuid: "0b83ec152b074f80af2848108d3af254" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 40357 } } peers { permanent_uuid: "b8e3bd4d570d422d8dd77fd3c581232b" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 45127 } }
I20260811 06:30:23.675500 9144 consensus_queue.cc:260] T 953060620da94a6381aba2a93be8e9cb P 7c0f7f24b8f0420a93c04bdf627b58e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f7f24b8f0420a93c04bdf627b58e1" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 44989 } } peers { permanent_uuid: "0b83ec152b074f80af2848108d3af254" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 40357 } } peers { permanent_uuid: "b8e3bd4d570d422d8dd77fd3c581232b" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 45127 } }
I20260811 06:30:23.681902 9142 ts_tablet_manager.cc:1428] T 953060620da94a6381aba2a93be8e9cb P 0b83ec152b074f80af2848108d3af254: Time spent starting tablet: real 0.008s user 0.002s sys 0.003s
I20260811 06:30:23.682564 8568 heartbeater.cc:503] Master 127.28.239.253:45951 was elected leader, sending a full tablet report...
I20260811 06:30:23.683148 9155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:23.681462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:40129 (local address 127.28.239.193:39229)
0811 06:30:23.682558 (+ 1096us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:23.682810 (+ 252us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:40129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":934,"thread_start_us":756,"threads_started":1}
I20260811 06:30:23.684823 9155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:23.683955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:40129 (local address 127.28.239.193:50335)
0811 06:30:23.684551 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:23.684642 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:40129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":505,"threads_started":1}
I20260811 06:30:23.686061 9144 ts_tablet_manager.cc:1428] T 953060620da94a6381aba2a93be8e9cb P 7c0f7f24b8f0420a93c04bdf627b58e1: Time spent starting tablet: real 0.012s user 0.005s sys 0.000s
I20260811 06:30:23.686399 9000 heartbeater.cc:503] Master 127.28.239.253:45951 was elected leader, sending a full tablet report...
I20260811 06:30:23.690496 9154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:23.678739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:40129 (local address 127.28.239.195:46811)
0811 06:30:23.689969 (+ 11230us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:23.690215 (+ 246us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:40129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":11114,"thread_start_us":10992,"threads_started":1}
I20260811 06:30:23.691145 9151 tablet_bootstrap.cc:492] T 953060620da94a6381aba2a93be8e9cb P b8e3bd4d570d422d8dd77fd3c581232b: Bootstrap starting.
I20260811 06:30:23.692289 9154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:23.691880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:40129 (local address 127.28.239.195:41445)
0811 06:30:23.692060 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:23.692154 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:40129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260811 06:30:23.692968 9151 tablet_bootstrap.cc:654] T 953060620da94a6381aba2a93be8e9cb P b8e3bd4d570d422d8dd77fd3c581232b: Neither blocks nor log segments found. Creating new log.
I20260811 06:30:23.695441 9151 tablet_bootstrap.cc:492] T 953060620da94a6381aba2a93be8e9cb P b8e3bd4d570d422d8dd77fd3c581232b: No bootstrap required, opened a new log
I20260811 06:30:23.695608 9151 ts_tablet_manager.cc:1397] T 953060620da94a6381aba2a93be8e9cb P b8e3bd4d570d422d8dd77fd3c581232b: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260811 06:30:23.696326 9151 raft_consensus.cc:348] T 953060620da94a6381aba2a93be8e9cb P b8e3bd4d570d422d8dd77fd3c581232b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f7f24b8f0420a93c04bdf627b58e1" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 44989 } } peers { permanent_uuid: "0b83ec152b074f80af2848108d3af254" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 40357 } } peers { permanent_uuid: "b8e3bd4d570d422d8dd77fd3c581232b" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 45127 } }
I20260811 06:30:23.696566 9151 raft_consensus.cc:374] T 953060620da94a6381aba2a93be8e9cb P b8e3bd4d570d422d8dd77fd3c581232b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:30:23.696687 9151 raft_consensus.cc:729] T 953060620da94a6381aba2a93be8e9cb P b8e3bd4d570d422d8dd77fd3c581232b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e3bd4d570d422d8dd77fd3c581232b, State: Initialized, Role: FOLLOWER
I20260811 06:30:23.696954 9151 consensus_queue.cc:260] T 953060620da94a6381aba2a93be8e9cb P b8e3bd4d570d422d8dd77fd3c581232b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f7f24b8f0420a93c04bdf627b58e1" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 44989 } } peers { permanent_uuid: "0b83ec152b074f80af2848108d3af254" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 40357 } } peers { permanent_uuid: "b8e3bd4d570d422d8dd77fd3c581232b" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 45127 } }
I20260811 06:30:23.697793 9151 ts_tablet_manager.cc:1428] T 953060620da94a6381aba2a93be8e9cb P b8e3bd4d570d422d8dd77fd3c581232b: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 06:30:23.699061 9162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:23.698101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:40129 (local address 127.28.239.194:39037)
0811 06:30:23.698696 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:23.698851 (+ 155us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:40129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":322,"threads_started":1}
I20260811 06:30:23.700088 9162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:23.699729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:40129 (local address 127.28.239.194:49159)
0811 06:30:23.699857 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:23.699940 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:40129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260811 06:30:23.700219 8783 heartbeater.cc:503] Master 127.28.239.253:45951 was elected leader, sending a full tablet report...
I20260811 06:30:23.865820 9160 raft_consensus.cc:482] T 953060620da94a6381aba2a93be8e9cb P b8e3bd4d570d422d8dd77fd3c581232b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:30:23.866115 9160 raft_consensus.cc:504] T 953060620da94a6381aba2a93be8e9cb P b8e3bd4d570d422d8dd77fd3c581232b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f7f24b8f0420a93c04bdf627b58e1" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 44989 } } peers { permanent_uuid: "0b83ec152b074f80af2848108d3af254" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 40357 } } peers { permanent_uuid: "b8e3bd4d570d422d8dd77fd3c581232b" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 45127 } }
I20260811 06:30:23.867867 9160 leader_election.cc:288] T 953060620da94a6381aba2a93be8e9cb P b8e3bd4d570d422d8dd77fd3c581232b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c0f7f24b8f0420a93c04bdf627b58e1 (127.28.239.195:44989), 0b83ec152b074f80af2848108d3af254 (127.28.239.193:40357)
I20260811 06:30:23.875440 9162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:23.868110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.195:44989 (local address 127.28.239.194:53841)
0811 06:30:23.868394 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:23.868416 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:23.868591 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:23.869229 (+ 638us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:23.869233 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:23.869258 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:30:23.869598 (+ 340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:23.869615 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:23.870900 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:23.870907 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:23.872972 (+ 2065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:23.872988 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:23.874644 (+ 1656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:23.874650 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:23.874825 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:23.874851 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 06:30:23.874971 (+ 120us) client_negotiation.cc:240] Negotiation successful
0811 06:30:23.875122 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"mutex_wait_us":28}
I20260811 06:30:23.875417 9135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:23.868328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.194:53841 (local address 127.28.239.195:44989)
0811 06:30:23.868532 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:23.868538 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:23.868561 (+ 23us) server_negotiation.cc:407] Connection header received
0811 06:30:23.868809 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:23.868816 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:23.868905 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:23.868983 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:30:23.869772 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:23.870756 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:23.873168 (+ 2412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:23.874420 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:23.874613 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:23.874693 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:23.874999 (+ 306us) server_negotiation.cc:299] Negotiation successful
0811 06:30:23.875146 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 06:30:23.876709 8928 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "953060620da94a6381aba2a93be8e9cb" candidate_uuid: "b8e3bd4d570d422d8dd77fd3c581232b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f7f24b8f0420a93c04bdf627b58e1" is_pre_election: true
I20260811 06:30:23.876943 9140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:23.869182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.194:53323 (local address 127.28.239.193:40357)
0811 06:30:23.869412 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:23.869419 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:23.869442 (+ 23us) server_negotiation.cc:407] Connection header received
0811 06:30:23.869555 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:23.869561 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:23.869642 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:23.869745 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:30:23.870358 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:23.871465 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:23.874271 (+ 2806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:23.875604 (+ 1333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:23.875782 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:23.875881 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:23.876526 (+ 645us) server_negotiation.cc:299] Negotiation successful
0811 06:30:23.876685 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260811 06:30:23.877118 9163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:23.868344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.193:40357 (local address 127.28.239.194:53323)
0811 06:30:23.869083 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:23.869102 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:23.869219 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:23.869805 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:23.869809 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:23.869842 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:30:23.870174 (+ 332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:23.870186 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:23.871633 (+ 1447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:23.871640 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:23.874062 (+ 2422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:23.874078 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:23.876094 (+ 2016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:23.876101 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:23.876322 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:23.876361 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 06:30:23.876470 (+ 109us) client_negotiation.cc:240] Negotiation successful
0811 06:30:23.876613 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":618,"mutex_wait_us":80,"thread_start_us":413,"threads_started":1}
I20260811 06:30:23.877223 8928 raft_consensus.cc:2454] T 953060620da94a6381aba2a93be8e9cb P 7c0f7f24b8f0420a93c04bdf627b58e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8e3bd4d570d422d8dd77fd3c581232b in term 0.
I20260811 06:30:23.877959 8616 leader_election.cc:302] T 953060620da94a6381aba2a93be8e9cb P b8e3bd4d570d422d8dd77fd3c581232b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c0f7f24b8f0420a93c04bdf627b58e1, b8e3bd4d570d422d8dd77fd3c581232b; no voters:
I20260811 06:30:23.878223 8488 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "953060620da94a6381aba2a93be8e9cb" candidate_uuid: "b8e3bd4d570d422d8dd77fd3c581232b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b83ec152b074f80af2848108d3af254" is_pre_election: true
I20260811 06:30:23.878451 9160 raft_consensus.cc:2781] T 953060620da94a6381aba2a93be8e9cb P b8e3bd4d570d422d8dd77fd3c581232b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:30:23.878587 9160 raft_consensus.cc:482] T 953060620da94a6381aba2a93be8e9cb P b8e3bd4d570d422d8dd77fd3c581232b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:30:23.878577 8488 raft_consensus.cc:2454] T 953060620da94a6381aba2a93be8e9cb P 0b83ec152b074f80af2848108d3af254 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8e3bd4d570d422d8dd77fd3c581232b in term 0.
I20260811 06:30:23.878687 9160 raft_consensus.cc:3037] T 953060620da94a6381aba2a93be8e9cb P b8e3bd4d570d422d8dd77fd3c581232b [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:30:23.882274 9160 raft_consensus.cc:504] T 953060620da94a6381aba2a93be8e9cb P b8e3bd4d570d422d8dd77fd3c581232b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f7f24b8f0420a93c04bdf627b58e1" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 44989 } } peers { permanent_uuid: "0b83ec152b074f80af2848108d3af254" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 40357 } } peers { permanent_uuid: "b8e3bd4d570d422d8dd77fd3c581232b" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 45127 } }
I20260811 06:30:23.883105 9160 leader_election.cc:288] T 953060620da94a6381aba2a93be8e9cb P b8e3bd4d570d422d8dd77fd3c581232b [CANDIDATE]: Term 1 election: Requested vote from peers 7c0f7f24b8f0420a93c04bdf627b58e1 (127.28.239.195:44989), 0b83ec152b074f80af2848108d3af254 (127.28.239.193:40357)
I20260811 06:30:23.883656 8488 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "953060620da94a6381aba2a93be8e9cb" candidate_uuid: "b8e3bd4d570d422d8dd77fd3c581232b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b83ec152b074f80af2848108d3af254"
I20260811 06:30:23.883688 8928 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "953060620da94a6381aba2a93be8e9cb" candidate_uuid: "b8e3bd4d570d422d8dd77fd3c581232b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f7f24b8f0420a93c04bdf627b58e1"
I20260811 06:30:23.883885 8488 raft_consensus.cc:3037] T 953060620da94a6381aba2a93be8e9cb P 0b83ec152b074f80af2848108d3af254 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:30:23.883929 8928 raft_consensus.cc:3037] T 953060620da94a6381aba2a93be8e9cb P 7c0f7f24b8f0420a93c04bdf627b58e1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:30:23.886119 8488 raft_consensus.cc:2454] T 953060620da94a6381aba2a93be8e9cb P 0b83ec152b074f80af2848108d3af254 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8e3bd4d570d422d8dd77fd3c581232b in term 1.
I20260811 06:30:23.886648 8610 leader_election.cc:302] T 953060620da94a6381aba2a93be8e9cb P b8e3bd4d570d422d8dd77fd3c581232b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b83ec152b074f80af2848108d3af254, b8e3bd4d570d422d8dd77fd3c581232b; no voters:
I20260811 06:30:23.887002 9160 raft_consensus.cc:2781] T 953060620da94a6381aba2a93be8e9cb P b8e3bd4d570d422d8dd77fd3c581232b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:30:23.887223 8928 raft_consensus.cc:2454] T 953060620da94a6381aba2a93be8e9cb P 7c0f7f24b8f0420a93c04bdf627b58e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8e3bd4d570d422d8dd77fd3c581232b in term 1.
I20260811 06:30:23.887935 9160 raft_consensus.cc:686] T 953060620da94a6381aba2a93be8e9cb P b8e3bd4d570d422d8dd77fd3c581232b [term 1 LEADER]: Becoming Leader. State: Replica: b8e3bd4d570d422d8dd77fd3c581232b, State: Running, Role: LEADER
I20260811 06:30:23.888404 9160 consensus_queue.cc:237] T 953060620da94a6381aba2a93be8e9cb P b8e3bd4d570d422d8dd77fd3c581232b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f7f24b8f0420a93c04bdf627b58e1" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 44989 } } peers { permanent_uuid: "0b83ec152b074f80af2848108d3af254" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 40357 } } peers { permanent_uuid: "b8e3bd4d570d422d8dd77fd3c581232b" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 45127 } }
I20260811 06:30:23.889564 9163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:23.888995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:40129 (local address 127.28.239.194:39131)
0811 06:30:23.889198 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:23.889356 (+ 158us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:40129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260811 06:30:23.892436 7994 catalog_manager.cc:5495] T 953060620da94a6381aba2a93be8e9cb P b8e3bd4d570d422d8dd77fd3c581232b reported cstate change: term changed from 0 to 1, leader changed from <none> to b8e3bd4d570d422d8dd77fd3c581232b (127.28.239.194). New cstate: current_term: 1 leader_uuid: "b8e3bd4d570d422d8dd77fd3c581232b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f7f24b8f0420a93c04bdf627b58e1" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b83ec152b074f80af2848108d3af254" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 40357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8e3bd4d570d422d8dd77fd3c581232b" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 45127 } health_report { overall_health: HEALTHY } } }
I20260811 06:30:23.962970 9141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:23.957452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33720 (local address 127.28.239.194:45127)
0811 06:30:23.957715 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:23.957720 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:23.957740 (+ 20us) server_negotiation.cc:407] Connection header received
0811 06:30:23.957879 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:23.957883 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:23.957968 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:23.958118 (+ 150us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:30:23.958479 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:23.959212 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:23.960533 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:23.961063 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:23.961184 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:23.961601 (+ 417us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:30:23.962250 (+ 649us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:30:23.962386 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:23.962507 (+ 121us) server_negotiation.cc:299] Negotiation successful
0811 06:30:23.962586 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260811 06:30:23.967427 8488 raft_consensus.cc:1261] T 953060620da94a6381aba2a93be8e9cb P 0b83ec152b074f80af2848108d3af254 [term 1 FOLLOWER]: Refusing update from remote peer b8e3bd4d570d422d8dd77fd3c581232b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:30:23.967881 8928 raft_consensus.cc:1261] T 953060620da94a6381aba2a93be8e9cb P 7c0f7f24b8f0420a93c04bdf627b58e1 [term 1 FOLLOWER]: Refusing update from remote peer b8e3bd4d570d422d8dd77fd3c581232b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:30:23.968217 9165 consensus_queue.cc:1035] T 953060620da94a6381aba2a93be8e9cb P b8e3bd4d570d422d8dd77fd3c581232b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b83ec152b074f80af2848108d3af254" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 40357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:30:23.968760 9160 consensus_queue.cc:1035] T 953060620da94a6381aba2a93be8e9cb P b8e3bd4d570d422d8dd77fd3c581232b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c0f7f24b8f0420a93c04bdf627b58e1" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:30:23.971967 9156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:23.971054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:40129 (local address 127.28.239.193:37701)
0811 06:30:23.971458 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:23.971693 (+ 235us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:40129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202}
I20260811 06:30:23.972615 9154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:23.971088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:40129 (local address 127.28.239.195:33557)
0811 06:30:23.972265 (+ 1177us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:23.972390 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:40129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1053}
I20260811 06:30:23.976531 9171 mvcc.cc:205] Tried to move back new op lower bound from 7317216558963654656 to 7317216558652379136. Current Snapshot: MvccSnapshot[applied={T|T < 7317216558963654656}]
I20260811 06:30:23.978832 9156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:23.977827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:40129 (local address 127.28.239.193:58171)
0811 06:30:23.978481 (+ 654us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:23.978608 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:40129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":495}
I20260811 06:30:23.979482 9173 mvcc.cc:205] Tried to move back new op lower bound from 7317216558963654656 to 7317216558652379136. Current Snapshot: MvccSnapshot[applied={T|T < 7317216558963654656}]
I20260811 06:30:23.981143 9175 mvcc.cc:205] Tried to move back new op lower bound from 7317216558963654656 to 7317216558652379136. Current Snapshot: MvccSnapshot[applied={T|T < 7317216558963654656}]
I20260811 06:30:23.982573 9154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:23.982101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:40129 (local address 127.28.239.195:46737)
0811 06:30:23.982339 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:23.982425 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:40129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137}
I20260811 06:30:23.984443 9163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:23.979333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:40129 (local address 127.28.239.194:49589)
0811 06:30:23.984116 (+ 4783us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:23.984250 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:40129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4650}
I20260811 06:30:23.993597 7992 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55950:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:30:23.993952 7992 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55950:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:30:23.994341 9163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:23.993160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:40129 (local address 127.28.239.194:46165)
0811 06:30:23.993511 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:23.993682 (+ 171us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:40129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124}
I20260811 06:30:24.001610 7992 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 81d907d1c7ce487da25ae51b57e094e3: Sending DeleteTablet for 3 replicas of tablet 953060620da94a6381aba2a93be8e9cb
I20260811 06:30:24.002609 8896 tablet_service.cc:1514] Processing DeleteTablet for tablet 953060620da94a6381aba2a93be8e9cb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:30:23 UTC) from {username='oryx', principal='oryx/127.28.239.253@KRBTEST.COM'} at 127.0.0.1:43388
I20260811 06:30:24.002920 8460 tablet_service.cc:1514] Processing DeleteTablet for tablet 953060620da94a6381aba2a93be8e9cb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:30:23 UTC) from {username='oryx', principal='oryx/127.28.239.253@KRBTEST.COM'} at 127.0.0.1:40584
I20260811 06:30:24.003573 9187 tablet_replica.cc:330] T 953060620da94a6381aba2a93be8e9cb P 7c0f7f24b8f0420a93c04bdf627b58e1: stopping tablet replica
I20260811 06:30:24.003793 9189 tablet_replica.cc:330] T 953060620da94a6381aba2a93be8e9cb P 0b83ec152b074f80af2848108d3af254: stopping tablet replica
I20260811 06:30:24.003904 9187 raft_consensus.cc:2229] T 953060620da94a6381aba2a93be8e9cb P 7c0f7f24b8f0420a93c04bdf627b58e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:30:24.004005 9189 raft_consensus.cc:2229] T 953060620da94a6381aba2a93be8e9cb P 0b83ec152b074f80af2848108d3af254 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:30:24.004267 8678 tablet_service.cc:1514] Processing DeleteTablet for tablet 953060620da94a6381aba2a93be8e9cb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:30:23 UTC) from {username='oryx', principal='oryx/127.28.239.253@KRBTEST.COM'} at 127.0.0.1:33718
I20260811 06:30:24.004333 9189 raft_consensus.cc:2258] T 953060620da94a6381aba2a93be8e9cb P 0b83ec152b074f80af2848108d3af254 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:30:24.004331 9187 raft_consensus.cc:2258] T 953060620da94a6381aba2a93be8e9cb P 7c0f7f24b8f0420a93c04bdf627b58e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:30:24.005926 9187 ts_tablet_manager.cc:1905] T 953060620da94a6381aba2a93be8e9cb P 7c0f7f24b8f0420a93c04bdf627b58e1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:30:24.005940 9189 ts_tablet_manager.cc:1905] T 953060620da94a6381aba2a93be8e9cb P 0b83ec152b074f80af2848108d3af254: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:30:24.011042 9190 tablet_replica.cc:330] T 953060620da94a6381aba2a93be8e9cb P b8e3bd4d570d422d8dd77fd3c581232b: stopping tablet replica
I20260811 06:30:24.011291 9190 raft_consensus.cc:2229] T 953060620da94a6381aba2a93be8e9cb P b8e3bd4d570d422d8dd77fd3c581232b [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:30:24.011950 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 8266
I20260811 06:30:24.011999 9190 raft_consensus.cc:2258] T 953060620da94a6381aba2a93be8e9cb P b8e3bd4d570d422d8dd77fd3c581232b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:30:24.016721 9189 ts_tablet_manager.cc:1918] T 953060620da94a6381aba2a93be8e9cb P 0b83ec152b074f80af2848108d3af254: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:30:24.016909 9189 log.cc:1196] T 953060620da94a6381aba2a93be8e9cb P 0b83ec152b074f80af2848108d3af254: Deleting WAL directory at /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-0/wal/wals/953060620da94a6381aba2a93be8e9cb
I20260811 06:30:24.017551 9189 ts_tablet_manager.cc:1939] T 953060620da94a6381aba2a93be8e9cb P 0b83ec152b074f80af2848108d3af254: Deleting consensus metadata
I20260811 06:30:24.018527 9190 ts_tablet_manager.cc:1905] T 953060620da94a6381aba2a93be8e9cb P b8e3bd4d570d422d8dd77fd3c581232b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:30:24.019129 7967 catalog_manager.cc:4841] TS 0b83ec152b074f80af2848108d3af254 (127.28.239.193:40357): tablet 953060620da94a6381aba2a93be8e9cb (table test-table [id=a7b99412561a479dae90bad34a2cb5c2]) successfully deleted
I20260811 06:30:24.019690 9187 ts_tablet_manager.cc:1918] T 953060620da94a6381aba2a93be8e9cb P 7c0f7f24b8f0420a93c04bdf627b58e1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:30:24.020993 9187 log.cc:1196] T 953060620da94a6381aba2a93be8e9cb P 7c0f7f24b8f0420a93c04bdf627b58e1: Deleting WAL directory at /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-2/wal/wals/953060620da94a6381aba2a93be8e9cb
I20260811 06:30:24.021438 9187 ts_tablet_manager.cc:1939] T 953060620da94a6381aba2a93be8e9cb P 7c0f7f24b8f0420a93c04bdf627b58e1: Deleting consensus metadata
I20260811 06:30:24.022727 7973 catalog_manager.cc:4841] TS 7c0f7f24b8f0420a93c04bdf627b58e1 (127.28.239.195:44989): tablet 953060620da94a6381aba2a93be8e9cb (table test-table [id=a7b99412561a479dae90bad34a2cb5c2]) successfully deleted
I20260811 06:30:24.026607 9190 ts_tablet_manager.cc:1918] T 953060620da94a6381aba2a93be8e9cb P b8e3bd4d570d422d8dd77fd3c581232b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:30:24.026788 9190 log.cc:1196] T 953060620da94a6381aba2a93be8e9cb P b8e3bd4d570d422d8dd77fd3c581232b: Deleting WAL directory at /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786429755927508-29631-0/minicluster-data/ts-1/wal/wals/953060620da94a6381aba2a93be8e9cb
I20260811 06:30:24.027419 9190 ts_tablet_manager.cc:1939] T 953060620da94a6381aba2a93be8e9cb P b8e3bd4d570d422d8dd77fd3c581232b: Deleting consensus metadata
I20260811 06:30:24.028743 7972 catalog_manager.cc:4841] TS b8e3bd4d570d422d8dd77fd3c581232b (127.28.239.194:45127): tablet 953060620da94a6381aba2a93be8e9cb (table test-table [id=a7b99412561a479dae90bad34a2cb5c2]) successfully deleted
I20260811 06:30:24.041328 9140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:24.013231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40590 (local address 127.28.239.193:40357)
0811 06:30:24.013447 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:24.013455 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:24.027420 (+ 13965us) server_negotiation.cc:407] Connection header received
0811 06:30:24.027636 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:24.027647 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:24.027720 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:24.027837 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:24.029432 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:24.030286 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:24.031552 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:24.031998 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:24.032217 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:24.035116 (+ 2899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:24.035132 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:24.035151 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:30:24.035190 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:24.038308 (+ 3118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:24.039078 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:24.039087 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:24.039089 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:24.039226 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:24.039628 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:24.039634 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:24.039637 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:24.040047 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:24.040408 (+ 361us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:24.040694 (+ 286us) server_negotiation.cc:299] Negotiation successful
0811 06:30:24.040958 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 06:30:24.042135 8557 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:30:24.136766 9138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:24.136208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:40129 (local address 127.0.0.1:54616)
0811 06:30:24.136438 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:24.136563 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:40129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260811 06:30:24.138329 9138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:24.137895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:40129 (local address 127.0.0.1:54618)
0811 06:30:24.138070 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:24.138174 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:40129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260811 06:30:24.161000 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 8266
I20260811 06:30:24.197017 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 8587
I20260811 06:30:24.214988 9141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:24.197962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33728 (local address 127.28.239.194:45127)
0811 06:30:24.198178 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:24.198184 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:24.198212 (+ 28us) server_negotiation.cc:407] Connection header received
0811 06:30:24.198440 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:24.198445 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:24.198517 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:24.198606 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:24.200348 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:24.201262 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:24.203347 (+ 2085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:24.203783 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:24.203878 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:24.207353 (+ 3475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:24.207370 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:24.207388 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:30:24.207423 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:24.211416 (+ 3993us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:24.212263 (+ 847us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:24.212273 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:24.212277 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:24.212560 (+ 283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:24.213022 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:24.213027 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:24.213030 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:24.213604 (+ 574us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:24.213993 (+ 389us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:24.214296 (+ 303us) server_negotiation.cc:299] Negotiation successful
0811 06:30:24.214578 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260811 06:30:24.216007 8773 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:30:24.338176 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 8587
I20260811 06:30:24.365707 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 8802
I20260811 06:30:24.382476 9135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:24.366757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43394 (local address 127.28.239.195:44989)
0811 06:30:24.366971 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:24.366980 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:24.367007 (+ 27us) server_negotiation.cc:407] Connection header received
0811 06:30:24.367264 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:24.367270 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:24.367347 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:24.367473 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:24.369103 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:24.370084 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:24.371433 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:24.371877 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:24.372046 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:24.375485 (+ 3439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:24.375500 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:24.375518 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:30:24.375567 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:24.379414 (+ 3847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:24.380310 (+ 896us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:24.380318 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:24.380321 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:24.380455 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:24.380843 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:24.380846 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:24.380848 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:24.381250 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:24.381479 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:24.381925 (+ 446us) server_negotiation.cc:299] Negotiation successful
0811 06:30:24.382126 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 06:30:24.383239 8993 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:30:24.510809 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 8802
I20260811 06:30:24.545889 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 7772
I20260811 06:30:24.547217 8043 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:30:24.682068 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 7772
W20260811 06:30:24.714623 8264 connection.cc:535] server connection from 127.0.0.1:45264 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 06:30:24.715035 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 7936
I20260811 06:30:24.716128 8335 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:30:24.837898 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 7936
2026-08-11T06:30:24Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (10921 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:30:24 dist-test-slave-gkw7 krb5kdc[9201](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:30:24 dist-test-slave-gkw7 krb5kdc[9201](info): set up 2 sockets
Aug 11 06:30:24 dist-test-slave-gkw7 krb5kdc[9201](info): commencing operation
krb5kdc: starting...
W20260811 06:30:26.248514 29631 mini_kdc.cc:121] Time spent starting KDC: real 1.350s user 0.002s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/krb5kdc/test-user.keytab.
Aug 11 06:30:26 dist-test-slave-gkw7 krb5kdc[9201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429826, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:30:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:30:26Z Disabled control of system clock
WARNING: no policy specified for oryx/127.28.239.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.28.239.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.28.239.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/krb5kdc/oryx_127.28.239.254.keytab.
Entry for principal oryx/127.28.239.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/krb5kdc/oryx_127.28.239.254.keytab.
I20260811 06:30:26.537801 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:46215
--webserver_interface=127.28.239.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/krb5kdc/oryx_127.28.239.254.keytab
--principal=oryx/127.28.239.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:39513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.239.254:46215
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:30:26.891491 9220 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:30:26.891872 9220 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:30:26.892050 9220 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:30:26.899700 9220 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:30:26.899860 9220 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 06:30:26.899961 9220 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:30:26.900084 9220 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:30:26.900174 9220 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:30:26.900285 9220 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:30:26.910676 9220 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:39513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.239.254:46215
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/krb5kdc/oryx_127.28.239.254.keytab
--principal=oryx/127.28.239.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:46215
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.239.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:30:26.912096 9220 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:30:26.913592 9220 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:30:26.926045 9226 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:30:26.926098 9229 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:30:26.926119 9227 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:30:26.927170 9220 server_base.cc:1034] running on GCE node
I20260811 06:30:26.928017 9220 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:30:26.929705 9220 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:30:26.930970 9220 hybrid_clock.cc:648] HybridClock initialized: now 1786429826930905 us; error 45 us; skew 500 ppm
I20260811 06:30:26.931358 9220 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:30:26 dist-test-slave-gkw7 krb5kdc[9201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429826, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.239.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:30:26.934701 9220 init.cc:377] Logged in from keytab as oryx/127.28.239.254@KRBTEST.COM (short username oryx)
I20260811 06:30:26.936336 9220 webserver.cc:460] Webserver started at http://127.28.239.254:40177/ using document root <none> and password file <none>
I20260811 06:30:26.937088 9220 fs_manager.cc:359] Metadata directory not provided
I20260811 06:30:26.937233 9220 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:30:26.937638 9220 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:30:26.940748 9220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/minicluster-data/master-0/data/instance:
uuid: "2678652809584c6cb6c707b112de9a62"
format_stamp: "Formatted at 2026-08-11 06:30:26 on dist-test-slave-gkw7"
server_key: "5594daba328a35f0f585069a1b6e32db"
server_key_iv: "3b87cf1f388302b0911b54990204e9bb"
server_key_version: "encryptionkey@0"
I20260811 06:30:26.941671 9220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/minicluster-data/master-0/wal/instance:
uuid: "2678652809584c6cb6c707b112de9a62"
format_stamp: "Formatted at 2026-08-11 06:30:26 on dist-test-slave-gkw7"
server_key: "5594daba328a35f0f585069a1b6e32db"
server_key_iv: "3b87cf1f388302b0911b54990204e9bb"
server_key_version: "encryptionkey@0"
I20260811 06:30:26.947126 9220 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 06:30:26.950749 9240 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:26.952064 9220 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 06:30:26.952271 9220 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/minicluster-data/master-0/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/minicluster-data/master-0/wal
uuid: "2678652809584c6cb6c707b112de9a62"
format_stamp: "Formatted at 2026-08-11 06:30:26 on dist-test-slave-gkw7"
server_key: "5594daba328a35f0f585069a1b6e32db"
server_key_iv: "3b87cf1f388302b0911b54990204e9bb"
server_key_version: "encryptionkey@0"
I20260811 06:30:26.952513 9220 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:30:26.979157 9220 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:30:26.983395 9220 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:30:26.983698 9220 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:30:27.007225 9220 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.254:46215
I20260811 06:30:27.007263 9330 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.254:46215 every 8 connection(s)
I20260811 06:30:27.008652 9220 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
I20260811 06:30:27.012790 9335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:30:27.017015 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 9220
I20260811 06:30:27.017246 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/minicluster-data/master-0/wal/instance
I20260811 06:30:27.018200 29631 external_mini_cluster.cc:1428] Setting key 7fbef09018a01fdadfaf2cb0314418f1
I20260811 06:30:27.025364 9335 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2678652809584c6cb6c707b112de9a62: Bootstrap starting.
I20260811 06:30:27.028728 9335 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2678652809584c6cb6c707b112de9a62: Neither blocks nor log segments found. Creating new log.
I20260811 06:30:27.029894 9335 log.cc:824] T 00000000000000000000000000000000 P 2678652809584c6cb6c707b112de9a62: Log is configured to *not* fsync() on all Append() calls
I20260811 06:30:27.033654 9335 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2678652809584c6cb6c707b112de9a62: No bootstrap required, opened a new log
Aug 11 06:30:27 dist-test-slave-gkw7 krb5kdc[9201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429826, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM
I20260811 06:30:27.043224 9335 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2678652809584c6cb6c707b112de9a62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2678652809584c6cb6c707b112de9a62" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 46215 } }
I20260811 06:30:27.043617 9335 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2678652809584c6cb6c707b112de9a62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:30:27.043752 9335 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2678652809584c6cb6c707b112de9a62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2678652809584c6cb6c707b112de9a62, State: Initialized, Role: FOLLOWER
I20260811 06:30:27.044561 9339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:27.020574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33926 (local address 127.28.239.254:46215)
0811 06:30:27.021505 (+ 931us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:27.021519 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:27.021636 (+ 117us) server_negotiation.cc:407] Connection header received
0811 06:30:27.023995 (+ 2359us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:27.024034 (+ 39us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:27.024549 (+ 515us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:27.025025 (+ 476us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:27.026641 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:27.028026 (+ 1385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:27.029463 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:27.030300 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:27.030502 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:27.035785 (+ 5283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:27.035794 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:27.035811 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:30:27.035864 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:27.039589 (+ 3725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:27.040491 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:27.040499 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:27.040506 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:27.040742 (+ 236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:27.041202 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:27.041205 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:27.041208 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:27.041961 (+ 753us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:27.042256 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:27.042729 (+ 473us) server_negotiation.cc:299] Negotiation successful
0811 06:30:27.043226 (+ 497us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":680,"thread_start_us":419,"threads_started":1}
I20260811 06:30:27.044442 9335 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2678652809584c6cb6c707b112de9a62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2678652809584c6cb6c707b112de9a62" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 46215 } }
I20260811 06:30:27.044880 9335 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2678652809584c6cb6c707b112de9a62 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:30:27.045068 9335 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2678652809584c6cb6c707b112de9a62 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:30:27.045353 9335 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2678652809584c6cb6c707b112de9a62 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:30:27.048702 9335 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2678652809584c6cb6c707b112de9a62 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2678652809584c6cb6c707b112de9a62" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 46215 } }
I20260811 06:30:27.049328 9335 leader_election.cc:302] T 00000000000000000000000000000000 P 2678652809584c6cb6c707b112de9a62 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2678652809584c6cb6c707b112de9a62; no voters:
I20260811 06:30:27.050088 9335 leader_election.cc:288] T 00000000000000000000000000000000 P 2678652809584c6cb6c707b112de9a62 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:30:27.050244 9342 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2678652809584c6cb6c707b112de9a62 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:30:27.051527 9342 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2678652809584c6cb6c707b112de9a62 [term 1 LEADER]: Becoming Leader. State: Replica: 2678652809584c6cb6c707b112de9a62, State: Running, Role: LEADER
I20260811 06:30:27.052124 9342 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2678652809584c6cb6c707b112de9a62 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2678652809584c6cb6c707b112de9a62" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 46215 } }
I20260811 06:30:27.052444 9335 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2678652809584c6cb6c707b112de9a62 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:30:27.055634 9344 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2678652809584c6cb6c707b112de9a62 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2678652809584c6cb6c707b112de9a62. Latest consensus state: current_term: 1 leader_uuid: "2678652809584c6cb6c707b112de9a62" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2678652809584c6cb6c707b112de9a62" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 46215 } } }
I20260811 06:30:27.055455 9343 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2678652809584c6cb6c707b112de9a62 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2678652809584c6cb6c707b112de9a62" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2678652809584c6cb6c707b112de9a62" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 46215 } } }
I20260811 06:30:27.056092 9343 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2678652809584c6cb6c707b112de9a62 [sys.catalog]: This master's current role is: LEADER
I20260811 06:30:27.056097 9344 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2678652809584c6cb6c707b112de9a62 [sys.catalog]: This master's current role is: LEADER
I20260811 06:30:27.062678 9355 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:30:27.068749 9355 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:30:27.079669 9355 catalog_manager.cc:1347] Generated new cluster ID: 1c98d8d974654c9f8f7e4b2f91b4a2ee
I20260811 06:30:27.079836 9355 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:30:27.104404 9355 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:30:27.106478 9355 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:30:27.125078 9355 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2678652809584c6cb6c707b112de9a62: Generated new TSK 0
I20260811 06:30:27.126366 9355 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.28.239.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.28.239.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.28.239.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/krb5kdc/oryx_127.28.239.193.keytab.
Entry for principal oryx/127.28.239.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/krb5kdc/oryx_127.28.239.193.keytab.
I20260811 06:30:27.197650 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.193:0
--local_ip_for_outbound_sockets=127.28.239.193
--webserver_interface=127.28.239.193
--webserver_port=0
--tserver_master_addrs=127.28.239.254:46215
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/krb5kdc/oryx_127.28.239.193.keytab
--principal=oryx/127.28.239.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:39513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:30:27.563542 9381 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:30:27.563898 9381 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:30:27.564073 9381 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:30:27.571723 9381 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:30:27.572104 9381 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:30:27.572299 9381 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 06:30:27.572456 9381 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.193
I20260811 06:30:27.582479 9381 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:39513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/krb5kdc/oryx_127.28.239.193.keytab
--principal=oryx/127.28.239.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.239.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.28.239.254:46215
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.193
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:30:27.584254 9381 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:30:27.586028 9381 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:30:27.600356 9390 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:30:27.600407 9394 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:30:27.600418 9391 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:30:27.601738 9381 server_base.cc:1034] running on GCE node
I20260811 06:30:27.602682 9381 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:30:27.603749 9381 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:30:27.604983 9381 hybrid_clock.cc:648] HybridClock initialized: now 1786429827604916 us; error 39 us; skew 500 ppm
I20260811 06:30:27.605357 9381 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:30:27 dist-test-slave-gkw7 krb5kdc[9201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429827, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.239.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:30:27.624747 9381 init.cc:377] Logged in from keytab as oryx/127.28.239.193@KRBTEST.COM (short username oryx)
I20260811 06:30:27.626714 9381 webserver.cc:460] Webserver started at http://127.28.239.193:45203/ using document root <none> and password file <none>
I20260811 06:30:27.627492 9381 fs_manager.cc:359] Metadata directory not provided
I20260811 06:30:27.627619 9381 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:30:27.628023 9381 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:30:27.631217 9381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/minicluster-data/ts-0/data/instance:
uuid: "57b2448bb01c461a964ee8ceca7f2315"
format_stamp: "Formatted at 2026-08-11 06:30:27 on dist-test-slave-gkw7"
server_key: "a29056f74dfc91d79a03c34318cbd183"
server_key_iv: "a2cc4c610b3332a137bd9c2300442224"
server_key_version: "encryptionkey@0"
I20260811 06:30:27.631984 9381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/minicluster-data/ts-0/wal/instance:
uuid: "57b2448bb01c461a964ee8ceca7f2315"
format_stamp: "Formatted at 2026-08-11 06:30:27 on dist-test-slave-gkw7"
server_key: "a29056f74dfc91d79a03c34318cbd183"
server_key_iv: "a2cc4c610b3332a137bd9c2300442224"
server_key_version: "encryptionkey@0"
I20260811 06:30:27.637359 9381 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 06:30:27.641086 9405 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:27.642629 9381 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:30:27.642778 9381 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/minicluster-data/ts-0/wal
uuid: "57b2448bb01c461a964ee8ceca7f2315"
format_stamp: "Formatted at 2026-08-11 06:30:27 on dist-test-slave-gkw7"
server_key: "a29056f74dfc91d79a03c34318cbd183"
server_key_iv: "a2cc4c610b3332a137bd9c2300442224"
server_key_version: "encryptionkey@0"
I20260811 06:30:27.642976 9381 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:30:27.659072 9381 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:30:27.663441 9381 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:30:27.663682 9381 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:30:27.666746 9381 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:30:27.666862 9381 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:27.667006 9381 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:30:27.667219 9381 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
Aug 11 06:30:27 dist-test-slave-gkw7 krb5kdc[9201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429827, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.239.193@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM
I20260811 06:30:27.693585 9423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:27.667523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.193:43043 (local address 127.28.239.254:46215)
0811 06:30:27.670161 (+ 2638us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:27.670168 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:27.670194 (+ 26us) server_negotiation.cc:407] Connection header received
0811 06:30:27.671866 (+ 1672us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:27.671874 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:27.671994 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:27.672125 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:27.674690 (+ 2565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:27.675876 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:27.678760 (+ 2884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:27.679524 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:27.679719 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:27.685364 (+ 5645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:27.685374 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:27.685381 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:30:27.685444 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:27.689027 (+ 3583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:27.690688 (+ 1661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:27.690697 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:27.690703 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:27.690884 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:27.691589 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:27.691594 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:27.691596 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:27.691884 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:27.692067 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:27.692913 (+ 846us) server_negotiation.cc:299] Negotiation successful
0811 06:30:27.693173 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2491,"thread_start_us":2384,"threads_started":1}
I20260811 06:30:27.694921 9424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:27.668178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:46215 (local address 127.28.239.193:43043)
0811 06:30:27.669345 (+ 1167us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:27.669413 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:27.671532 (+ 2119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:27.672435 (+ 903us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:27.672451 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:27.673296 (+ 845us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:27.674415 (+ 1119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:27.674446 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:27.676054 (+ 1608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:27.676061 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:27.678558 (+ 2497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:27.678582 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:27.679829 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:27.679847 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:27.680146 (+ 299us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:27.681097 (+ 951us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:30:27.681162 (+ 65us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:27.684981 (+ 3819us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:30:27.689320 (+ 4339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:30:27.689332 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:30:27.689358 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:30:27.690482 (+ 1124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:30:27.691023 (+ 541us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:30:27.691028 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:30:27.691036 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:30:27.691403 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:30:27.692045 (+ 642us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:30:27.692054 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:30:27.692484 (+ 430us) client_negotiation.cc:769] Sending connection context
0811 06:30:27.692788 (+ 304us) client_negotiation.cc:240] Negotiation successful
0811 06:30:27.693420 (+ 632us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":765,"thread_start_us":466,"threads_started":1}
I20260811 06:30:27.718250 9423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:27.701856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.193:41997 (local address 127.28.239.254:46215)
0811 06:30:27.702078 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:27.702085 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:27.703117 (+ 1032us) server_negotiation.cc:407] Connection header received
0811 06:30:27.703493 (+ 376us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:27.703497 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:27.703581 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:27.703741 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:27.704766 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:27.705852 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:27.707327 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:27.707786 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:27.707965 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:27.710826 (+ 2861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:27.710832 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:27.710835 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:30:27.710870 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:27.714119 (+ 3249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:27.715257 (+ 1138us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:27.715262 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:27.715264 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:27.715397 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:27.715940 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:27.715946 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:27.715949 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:27.716413 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:27.716583 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:27.717705 (+ 1122us) server_negotiation.cc:299] Negotiation successful
0811 06:30:27.717916 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 06:30:27.718364 9535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:27.702119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:46215 (local address 127.28.239.193:41997)
0811 06:30:27.702966 (+ 847us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:27.702998 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:27.703308 (+ 310us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:27.703765 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:27.703769 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:27.704288 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:27.704596 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:27.704606 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:27.706005 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:27.706010 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:27.707172 (+ 1162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:27.707185 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:27.708039 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:27.708048 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:27.708242 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:27.709118 (+ 876us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:30:27.709159 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:27.710529 (+ 1370us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:30:27.714346 (+ 3817us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:30:27.714353 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:30:27.714363 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:30:27.715068 (+ 705us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:30:27.715594 (+ 526us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:30:27.715600 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:30:27.715606 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:30:27.715813 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:30:27.716530 (+ 717us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:30:27.716535 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:30:27.716830 (+ 295us) client_negotiation.cc:769] Sending connection context
0811 06:30:27.717030 (+ 200us) client_negotiation.cc:240] Negotiation successful
0811 06:30:27.717439 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":649,"thread_start_us":470,"threads_started":1}
I20260811 06:30:27.728121 9381 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.193:42829
I20260811 06:30:27.728163 9605 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.193:42829 every 8 connection(s)
I20260811 06:30:27.729593 9381 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
I20260811 06:30:27.732201 9606 heartbeater.cc:347] Connected to a master server at 127.28.239.254:46215
I20260811 06:30:27.732414 9606 heartbeater.cc:464] Registering TS with master...
I20260811 06:30:27.733099 9606 heartbeater.cc:511] Master 127.28.239.254:46215 requested a full tablet report, sending...
I20260811 06:30:27.735345 9267 ts_manager.cc:194] Registered new tserver with Master: 57b2448bb01c461a964ee8ceca7f2315 (127.28.239.193:42829)
I20260811 06:30:27.736819 9267 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.28.239.193@KRBTEST.COM'} at 127.28.239.193:43043
I20260811 06:30:27.739389 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 9381
I20260811 06:30:27.739596 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/minicluster-data/ts-0/wal/instance
I20260811 06:30:27.740289 29631 external_mini_cluster.cc:1428] Setting key 88ba7cdd67d6bbfdb029e96932e1fba9
WARNING: no policy specified for oryx/127.28.239.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.28.239.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.28.239.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/krb5kdc/oryx_127.28.239.194.keytab.
Entry for principal oryx/127.28.239.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/krb5kdc/oryx_127.28.239.194.keytab.
I20260811 06:30:27.804777 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.194:0
--local_ip_for_outbound_sockets=127.28.239.194
--webserver_interface=127.28.239.194
--webserver_port=0
--tserver_master_addrs=127.28.239.254:46215
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/krb5kdc/oryx_127.28.239.194.keytab
--principal=oryx/127.28.239.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:39513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:30:28.164552 9613 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:30:28.164856 9613 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:30:28.165035 9613 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:30:28.172415 9613 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:30:28.172538 9613 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:30:28.172700 9613 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 06:30:28.172825 9613 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.194
I20260811 06:30:28.183048 9613 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:39513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/krb5kdc/oryx_127.28.239.194.keytab
--principal=oryx/127.28.239.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.239.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.28.239.254:46215
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.194
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:30:28.184515 9613 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:30:28.186118 9613 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:30:28.200217 9618 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:30:28.200265 9624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:30:28.200304 9622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:30:28.201285 9613 server_base.cc:1034] running on GCE node
I20260811 06:30:28.202082 9613 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:30:28.203145 9613 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:30:28.204383 9613 hybrid_clock.cc:648] HybridClock initialized: now 1786429828204318 us; error 38 us; skew 500 ppm
I20260811 06:30:28.204720 9613 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:30:28 dist-test-slave-gkw7 krb5kdc[9201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429828, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.239.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:30:28.208402 9613 init.cc:377] Logged in from keytab as oryx/127.28.239.194@KRBTEST.COM (short username oryx)
I20260811 06:30:28.210172 9613 webserver.cc:460] Webserver started at http://127.28.239.194:38673/ using document root <none> and password file <none>
I20260811 06:30:28.210930 9613 fs_manager.cc:359] Metadata directory not provided
I20260811 06:30:28.211088 9613 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:30:28.211452 9613 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:30:28.214844 9613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/minicluster-data/ts-1/data/instance:
uuid: "2e276abb6ae14f2aab047be5cb9cb373"
format_stamp: "Formatted at 2026-08-11 06:30:28 on dist-test-slave-gkw7"
server_key: "b15f9c57063c1ce6f6fde4c6c07945fa"
server_key_iv: "36d06b6e04b0140d56f0128fe0ce9fc2"
server_key_version: "encryptionkey@0"
I20260811 06:30:28.215698 9613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/minicluster-data/ts-1/wal/instance:
uuid: "2e276abb6ae14f2aab047be5cb9cb373"
format_stamp: "Formatted at 2026-08-11 06:30:28 on dist-test-slave-gkw7"
server_key: "b15f9c57063c1ce6f6fde4c6c07945fa"
server_key_iv: "36d06b6e04b0140d56f0128fe0ce9fc2"
server_key_version: "encryptionkey@0"
I20260811 06:30:28.221398 9613 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260811 06:30:28.225244 9635 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:28.226768 9613 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:30:28.226976 9613 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/minicluster-data/ts-1/wal
uuid: "2e276abb6ae14f2aab047be5cb9cb373"
format_stamp: "Formatted at 2026-08-11 06:30:28 on dist-test-slave-gkw7"
server_key: "b15f9c57063c1ce6f6fde4c6c07945fa"
server_key_iv: "36d06b6e04b0140d56f0128fe0ce9fc2"
server_key_version: "encryptionkey@0"
I20260811 06:30:28.227249 9613 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:30:28.253800 9613 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:30:28.258876 9613 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:30:28.259148 9613 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:30:28.262006 9613 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:30:28.262167 9613 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:28.262305 9613 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:30:28.262377 9613 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 06:30:28 dist-test-slave-gkw7 krb5kdc[9201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429828, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.239.194@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM
I20260811 06:30:28.288198 9652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:28.262903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.194:40447 (local address 127.28.239.254:46215)
0811 06:30:28.264641 (+ 1738us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:28.264646 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:28.265373 (+ 727us) server_negotiation.cc:407] Connection header received
0811 06:30:28.268033 (+ 2660us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:28.268041 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:28.268137 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:28.268264 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:28.270713 (+ 2449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:28.272082 (+ 1369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:28.274747 (+ 2665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:28.275235 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:28.275394 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:28.281069 (+ 5675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:28.281075 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:28.281078 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:30:28.281113 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:28.284836 (+ 3723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:28.285681 (+ 845us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:28.285692 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:28.285695 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:28.285818 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:28.286312 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:28.286321 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:28.286323 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:28.286591 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:28.286786 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:28.287547 (+ 761us) server_negotiation.cc:299] Negotiation successful
0811 06:30:28.287810 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1616,"thread_start_us":1490,"threads_started":1}
I20260811 06:30:28.289659 9654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:28.263878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:46215 (local address 127.28.239.194:40447)
0811 06:30:28.265139 (+ 1261us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:28.265205 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:28.267493 (+ 2288us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:28.268531 (+ 1038us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:28.268542 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:28.269417 (+ 875us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:28.270434 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:28.270464 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:28.272301 (+ 1837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:28.272312 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:28.274548 (+ 2236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:28.274570 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:28.275553 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:28.275565 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:28.275962 (+ 397us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:28.276934 (+ 972us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:30:28.276977 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:28.280792 (+ 3815us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:30:28.284999 (+ 4207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:30:28.285009 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:30:28.285027 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:30:28.285483 (+ 456us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:30:28.285947 (+ 464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:30:28.285952 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:30:28.285959 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:30:28.286179 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:30:28.286742 (+ 563us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:30:28.286750 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:30:28.287154 (+ 404us) client_negotiation.cc:769] Sending connection context
0811 06:30:28.287438 (+ 284us) client_negotiation.cc:240] Negotiation successful
0811 06:30:28.287999 (+ 561us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":995,"thread_start_us":766,"threads_started":1}
I20260811 06:30:28.312708 9731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:28.296138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:46215 (local address 127.28.239.194:44911)
0811 06:30:28.296785 (+ 647us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:28.296804 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:28.296982 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:28.297626 (+ 644us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:28.297635 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:28.298749 (+ 1114us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:28.299405 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:28.299425 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:28.300801 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:28.300810 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:28.302552 (+ 1742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:28.302576 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:28.303616 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:28.303623 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:28.303807 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:28.304525 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:30:28.304542 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:28.305310 (+ 768us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:30:28.309704 (+ 4394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:30:28.309710 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:30:28.309715 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:30:28.310218 (+ 503us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:30:28.310679 (+ 461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:30:28.310683 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:30:28.310685 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:30:28.310814 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:30:28.311681 (+ 867us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:30:28.311685 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:30:28.311928 (+ 243us) client_negotiation.cc:769] Sending connection context
0811 06:30:28.312096 (+ 168us) client_negotiation.cc:240] Negotiation successful
0811 06:30:28.312370 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":364,"threads_started":1}
I20260811 06:30:28.313100 9652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:28.295987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.194:44911 (local address 127.28.239.254:46215)
0811 06:30:28.296187 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:28.296191 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:28.296901 (+ 710us) server_negotiation.cc:407] Connection header received
0811 06:30:28.297227 (+ 326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:28.297236 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:28.297344 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:28.297464 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:28.299622 (+ 2158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:28.300559 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:28.302767 (+ 2208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:28.303396 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:28.303545 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:28.305513 (+ 1968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:28.305518 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:28.305521 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:30:28.305614 (+ 93us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:28.309507 (+ 3893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:28.310361 (+ 854us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:28.310367 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:28.310370 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:28.310534 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:28.310943 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:28.310950 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:28.310954 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:28.311434 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:28.311612 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:28.312294 (+ 682us) server_negotiation.cc:299] Negotiation successful
0811 06:30:28.312631 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 06:30:28.322163 9613 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.194:37681
I20260811 06:30:28.322270 9811 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.194:37681 every 8 connection(s)
I20260811 06:30:28.323459 9613 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
I20260811 06:30:28.325845 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 9613
I20260811 06:30:28.326066 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/minicluster-data/ts-1/wal/instance
I20260811 06:30:28.326107 9812 heartbeater.cc:347] Connected to a master server at 127.28.239.254:46215
I20260811 06:30:28.326457 9812 heartbeater.cc:464] Registering TS with master...
I20260811 06:30:28.326735 29631 external_mini_cluster.cc:1428] Setting key 9b75b67d2c1636ccdcd7ceecea536fd0
I20260811 06:30:28.327270 9812 heartbeater.cc:511] Master 127.28.239.254:46215 requested a full tablet report, sending...
I20260811 06:30:28.329018 9265 ts_manager.cc:194] Registered new tserver with Master: 2e276abb6ae14f2aab047be5cb9cb373 (127.28.239.194:37681)
I20260811 06:30:28.330248 9265 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.28.239.194@KRBTEST.COM'} at 127.28.239.194:40447
WARNING: no policy specified for oryx/127.28.239.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.28.239.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.28.239.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/krb5kdc/oryx_127.28.239.195.keytab.
Entry for principal oryx/127.28.239.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/krb5kdc/oryx_127.28.239.195.keytab.
I20260811 06:30:28.389880 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.195:0
--local_ip_for_outbound_sockets=127.28.239.195
--webserver_interface=127.28.239.195
--webserver_port=0
--tserver_master_addrs=127.28.239.254:46215
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/krb5kdc/oryx_127.28.239.195.keytab
--principal=oryx/127.28.239.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:39513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 06:30:28.739877 9606 heartbeater.cc:503] Master 127.28.239.254:46215 was elected leader, sending a full tablet report...
W20260811 06:30:28.746951 9816 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:30:28.747274 9816 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:30:28.747454 9816 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:30:28.755793 9816 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:30:28.755959 9816 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:30:28.756129 9816 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 06:30:28.756270 9816 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.195
I20260811 06:30:28.766484 9816 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:39513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/krb5kdc/oryx_127.28.239.195.keytab
--principal=oryx/127.28.239.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.239.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.28.239.254:46215
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.195
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:30:28.767864 9816 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:30:28.769241 9816 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:30:28.783004 9822 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:30:28.782986 9827 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:30:28.782995 9824 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:30:28.784294 9816 server_base.cc:1034] running on GCE node
I20260811 06:30:28.784965 9816 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:30:28.786091 9816 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:30:28.787318 9816 hybrid_clock.cc:648] HybridClock initialized: now 1786429828787230 us; error 62 us; skew 500 ppm
I20260811 06:30:28.787640 9816 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:30:28 dist-test-slave-gkw7 krb5kdc[9201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429828, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.239.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:30:28.791278 9816 init.cc:377] Logged in from keytab as oryx/127.28.239.195@KRBTEST.COM (short username oryx)
I20260811 06:30:28.792865 9816 webserver.cc:460] Webserver started at http://127.28.239.195:45805/ using document root <none> and password file <none>
I20260811 06:30:28.793704 9816 fs_manager.cc:359] Metadata directory not provided
I20260811 06:30:28.793825 9816 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:30:28.794176 9816 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:30:28.797374 9816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/minicluster-data/ts-2/data/instance:
uuid: "34c9ac0ce67d4df5863681360f27c4ac"
format_stamp: "Formatted at 2026-08-11 06:30:28 on dist-test-slave-gkw7"
server_key: "6b23800fb8ec241023a36d4daa7c8174"
server_key_iv: "5d3222b3d6b5c034f18a5c1d5bb3b5a3"
server_key_version: "encryptionkey@0"
I20260811 06:30:28.798355 9816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/minicluster-data/ts-2/wal/instance:
uuid: "34c9ac0ce67d4df5863681360f27c4ac"
format_stamp: "Formatted at 2026-08-11 06:30:28 on dist-test-slave-gkw7"
server_key: "6b23800fb8ec241023a36d4daa7c8174"
server_key_iv: "5d3222b3d6b5c034f18a5c1d5bb3b5a3"
server_key_version: "encryptionkey@0"
I20260811 06:30:28.803882 9816 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 06:30:28.807585 9840 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:28.808954 9816 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 06:30:28.809149 9816 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/minicluster-data/ts-2/wal
uuid: "34c9ac0ce67d4df5863681360f27c4ac"
format_stamp: "Formatted at 2026-08-11 06:30:28 on dist-test-slave-gkw7"
server_key: "6b23800fb8ec241023a36d4daa7c8174"
server_key_iv: "5d3222b3d6b5c034f18a5c1d5bb3b5a3"
server_key_version: "encryptionkey@0"
I20260811 06:30:28.809383 9816 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:30:28.824528 9816 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:30:28.829120 9816 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:30:28.829406 9816 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:30:28.832320 9816 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:30:28.832513 9816 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:28.832684 9816 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:30:28.832773 9816 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 06:30:28 dist-test-slave-gkw7 krb5kdc[9201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429828, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.239.195@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM
I20260811 06:30:28.859771 9852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:28.832935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.195:55261 (local address 127.28.239.254:46215)
0811 06:30:28.833745 (+ 810us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:28.833751 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:28.834871 (+ 1120us) server_negotiation.cc:407] Connection header received
0811 06:30:28.836975 (+ 2104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:28.836981 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:28.837075 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:28.837232 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:28.840321 (+ 3089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:28.841918 (+ 1597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:28.843722 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:28.844492 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:28.844651 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:28.850570 (+ 5919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:28.850577 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:28.850585 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:30:28.850639 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:28.854930 (+ 4291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:28.856248 (+ 1318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:28.856253 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:28.856257 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:28.856418 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:28.857079 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:28.857084 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:28.857087 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:28.857557 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:28.857731 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:28.858951 (+ 1220us) server_negotiation.cc:299] Negotiation successful
0811 06:30:28.859229 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":675,"thread_start_us":515,"threads_started":1}
I20260811 06:30:28.861601 9854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:28.833671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:46215 (local address 127.28.239.195:55261)
0811 06:30:28.834664 (+ 993us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:28.834710 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:28.836667 (+ 1957us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:28.837755 (+ 1088us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:28.837771 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:28.838953 (+ 1182us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:28.840051 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:28.840088 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:28.842154 (+ 2066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:28.842160 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:28.843554 (+ 1394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:28.843569 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:28.844777 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:28.844789 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:28.845174 (+ 385us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:28.846140 (+ 966us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:30:28.846176 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:28.850259 (+ 4083us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:30:28.855132 (+ 4873us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:30:28.855146 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:30:28.855169 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:30:28.856096 (+ 927us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:30:28.856538 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:30:28.856546 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:30:28.856552 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:30:28.856903 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:30:28.857744 (+ 841us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:30:28.857757 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:30:28.858374 (+ 617us) client_negotiation.cc:769] Sending connection context
0811 06:30:28.858788 (+ 414us) client_negotiation.cc:240] Negotiation successful
0811 06:30:28.859535 (+ 747us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":682,"thread_start_us":434,"threads_started":1}
I20260811 06:30:28.876914 9816 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.195:37827
I20260811 06:30:28.877013 10012 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.195:37827 every 8 connection(s)
I20260811 06:30:28.878690 9816 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
I20260811 06:30:28.881472 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 9816
I20260811 06:30:28.881632 10013 heartbeater.cc:347] Connected to a master server at 127.28.239.254:46215
I20260811 06:30:28.881733 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/minicluster-data/ts-2/wal/instance
I20260811 06:30:28.881911 10013 heartbeater.cc:464] Registering TS with master...
I20260811 06:30:28.882287 29631 external_mini_cluster.cc:1428] Setting key 4109aa2592c60e3a098947678056ab5e
I20260811 06:30:28.882807 10013 heartbeater.cc:511] Master 127.28.239.254:46215 requested a full tablet report, sending...
I20260811 06:30:28.884819 9265 ts_manager.cc:194] Registered new tserver with Master: 34c9ac0ce67d4df5863681360f27c4ac (127.28.239.195:37827)
I20260811 06:30:28.886121 9265 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.28.239.195@KRBTEST.COM'} at 127.28.239.195:55261
I20260811 06:30:28.886221 29631 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 06:30:28.887140 9994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:28.869836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:46215 (local address 127.28.239.195:45267)
0811 06:30:28.870677 (+ 841us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:28.870693 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:28.870873 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:28.871472 (+ 599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:28.871478 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:28.872238 (+ 760us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:28.872859 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:28.872872 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:28.874449 (+ 1577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:28.874454 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:28.876404 (+ 1950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:28.876425 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:28.877520 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:28.877553 (+ 33us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:28.877753 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:28.878436 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:30:28.878458 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:28.880061 (+ 1603us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:30:28.884452 (+ 4391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:30:28.884458 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:30:28.884462 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:30:28.884846 (+ 384us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:30:28.885290 (+ 444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:30:28.885297 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:30:28.885303 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:30:28.885461 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:30:28.886173 (+ 712us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:30:28.886178 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:30:28.886371 (+ 193us) client_negotiation.cc:769] Sending connection context
0811 06:30:28.886519 (+ 148us) client_negotiation.cc:240] Negotiation successful
0811 06:30:28.886790 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":697,"thread_start_us":539,"threads_started":1}
I20260811 06:30:28.887892 9852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:28.869690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.195:45267 (local address 127.28.239.254:46215)
0811 06:30:28.869884 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:28.869888 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:28.870792 (+ 904us) server_negotiation.cc:407] Connection header received
0811 06:30:28.871037 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:28.871045 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:28.871155 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:28.871265 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:28.873022 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:28.874226 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:28.876587 (+ 2361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:28.877314 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:28.877416 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:28.880336 (+ 2920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:28.880341 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:28.880345 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:30:28.880405 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:28.884336 (+ 3931us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:28.884978 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:28.884984 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:28.884987 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:28.885151 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:28.885630 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:28.885634 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:28.885637 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:28.886017 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:28.886218 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:28.887456 (+ 1238us) server_negotiation.cc:299] Negotiation successful
0811 06:30:28.887665 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
Aug 11 06:30:28 dist-test-slave-gkw7 krb5kdc[9201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429828, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 06:30:28 dist-test-slave-gkw7 krb5kdc[9201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:30:28 dist-test-slave-gkw7 krb5kdc[9201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
W20260811 06:30:28.948910 9852 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:30:28.937271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33928 (local address 127.28.239.254:46215)
0811 06:30:28.937460 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:28.937465 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:28.937721 (+ 256us) server_negotiation.cc:407] Connection header received
0811 06:30:28.937893 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:28.937897 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:28.937968 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:28.938058 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:28.939506 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:28.940233 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:28.941469 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:28.942012 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:28.942147 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:28.948538 (+ 6391us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33928: BlockingRecv error: recv got EOF from 127.0.0.1:33928 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 11 06:30:28 dist-test-slave-gkw7 krb5kdc[9201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429828, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM
I20260811 06:30:28.965842 9852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:28.952138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33940 (local address 127.28.239.254:46215)
0811 06:30:28.952321 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:28.952325 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:28.952633 (+ 308us) server_negotiation.cc:407] Connection header received
0811 06:30:28.952777 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:28.952781 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:28.952844 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:28.952931 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:28.954155 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:28.954894 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:28.956178 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:28.956701 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:28.956801 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:28.960414 (+ 3613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:28.960419 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:28.960422 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:30:28.960457 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:28.963378 (+ 2921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:28.963985 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:28.963989 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:28.963991 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:28.964120 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:28.964508 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:28.964512 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:28.964514 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:28.964781 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:28.964975 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:28.965295 (+ 320us) server_negotiation.cc:299] Negotiation successful
0811 06:30:28.965517 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 06:30:28.970261 9265 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33940:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:30:28.977334 9265 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:30:29.014946 10045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:28.990580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51296 (local address 127.28.239.193:42829)
0811 06:30:29.004915 (+ 14335us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:29.004924 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:29.004958 (+ 34us) server_negotiation.cc:407] Connection header received
0811 06:30:29.005069 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:29.005078 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:29.005397 (+ 319us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:29.005662 (+ 265us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:30:29.006518 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:29.008206 (+ 1688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:29.011370 (+ 3164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:29.013000 (+ 1630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:29.013177 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:29.013633 (+ 456us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:29.014358 (+ 725us) server_negotiation.cc:299] Negotiation successful
0811 06:30:29.014614 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14155,"thread_start_us":13982,"threads_started":1}
I20260811 06:30:29.015540 10043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:28.990443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.195:37827 (local address 127.0.0.1:42686)
0811 06:30:29.005023 (+ 14580us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:29.005038 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:29.005974 (+ 936us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:29.007134 (+ 1160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:29.007139 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:29.007169 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:30:29.007873 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:29.007892 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:29.010100 (+ 2208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:29.010105 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:29.012395 (+ 2290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:29.012411 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:29.014627 (+ 2216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:29.014633 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:29.014795 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:29.014828 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 06:30:29.014895 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 06:30:29.015105 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14484,"mutex_wait_us":3,"thread_start_us":12476,"threads_started":1}
I20260811 06:30:29.016110 10044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:28.990240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.193:42829 (local address 127.0.0.1:51296)
0811 06:30:29.003393 (+ 13153us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:29.003444 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:29.003702 (+ 258us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:29.005834 (+ 2132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:29.005842 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:29.005876 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:30:29.006330 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:29.006342 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:29.008408 (+ 2066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:29.008415 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:29.011145 (+ 2730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:29.011167 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:29.013838 (+ 2671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:29.013844 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:29.014052 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:29.014127 (+ 75us) client_negotiation.cc:769] Sending connection context
0811 06:30:29.015511 (+ 1384us) client_negotiation.cc:240] Negotiation successful
0811 06:30:29.015685 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12822,"mutex_wait_us":27,"spinlock_wait_cycles":2816,"thread_start_us":14543,"threads_started":1}
I20260811 06:30:29.017061 10047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:29.005215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.194:37681 (local address 127.0.0.1:42208)
0811 06:30:29.005885 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:29.005905 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:29.006059 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:29.007307 (+ 1248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:29.007314 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:29.007350 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:30:29.008938 (+ 1588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:29.008950 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:29.010814 (+ 1864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:29.010820 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:29.013015 (+ 2195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:29.013029 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:29.016306 (+ 3277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:29.016313 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:29.016506 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:29.016539 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 06:30:29.016616 (+ 77us) client_negotiation.cc:240] Negotiation successful
0811 06:30:29.016740 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":310,"threads_started":1}
I20260811 06:30:29.017210 10046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:28.991044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42686 (local address 127.28.239.195:37827)
0811 06:30:29.004597 (+ 13553us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:29.004606 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:29.005181 (+ 575us) server_negotiation.cc:407] Connection header received
0811 06:30:29.006313 (+ 1132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:29.006321 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:29.006618 (+ 297us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:29.006820 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:30:29.008097 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:29.009945 (+ 1848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:29.012649 (+ 2704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:29.014368 (+ 1719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:29.016483 (+ 2115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:29.016658 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:29.016724 (+ 66us) server_negotiation.cc:299] Negotiation successful
0811 06:30:29.016870 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13305,"thread_start_us":13073,"threads_started":1}
I20260811 06:30:29.017876 10048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:29.005474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42208 (local address 127.28.239.194:37681)
0811 06:30:29.006523 (+ 1049us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:29.006531 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:29.006556 (+ 25us) server_negotiation.cc:407] Connection header received
0811 06:30:29.006662 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:29.006670 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:29.006879 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:29.007071 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:30:29.009124 (+ 2053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:29.010639 (+ 1515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:29.013268 (+ 2629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:29.015829 (+ 2561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:29.015989 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:29.016146 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:29.017375 (+ 1229us) server_negotiation.cc:299] Negotiation successful
0811 06:30:29.017576 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":879,"thread_start_us":736,"threads_started":1}
I20260811 06:30:29.020640 9490 tablet_service.cc:1467] Processing CreateTablet for tablet 99a3f5ed1bea49f1aef444fbd19f32d8 (DEFAULT_TABLE table=test-table [id=49f84001a1f549e09dfb1390f4b741fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:30:29.021450 9706 tablet_service.cc:1467] Processing CreateTablet for tablet 99a3f5ed1bea49f1aef444fbd19f32d8 (DEFAULT_TABLE table=test-table [id=49f84001a1f549e09dfb1390f4b741fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:30:29.021580 9910 tablet_service.cc:1467] Processing CreateTablet for tablet 99a3f5ed1bea49f1aef444fbd19f32d8 (DEFAULT_TABLE table=test-table [id=49f84001a1f549e09dfb1390f4b741fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:30:29.022562 9490 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99a3f5ed1bea49f1aef444fbd19f32d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:30:29.022951 9706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99a3f5ed1bea49f1aef444fbd19f32d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:30:29.024442 9910 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99a3f5ed1bea49f1aef444fbd19f32d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:30:29.036723 10049 tablet_bootstrap.cc:492] T 99a3f5ed1bea49f1aef444fbd19f32d8 P 2e276abb6ae14f2aab047be5cb9cb373: Bootstrap starting.
I20260811 06:30:29.040114 10049 tablet_bootstrap.cc:654] T 99a3f5ed1bea49f1aef444fbd19f32d8 P 2e276abb6ae14f2aab047be5cb9cb373: Neither blocks nor log segments found. Creating new log.
I20260811 06:30:29.041043 10050 tablet_bootstrap.cc:492] T 99a3f5ed1bea49f1aef444fbd19f32d8 P 57b2448bb01c461a964ee8ceca7f2315: Bootstrap starting.
I20260811 06:30:29.041597 10049 log.cc:824] T 99a3f5ed1bea49f1aef444fbd19f32d8 P 2e276abb6ae14f2aab047be5cb9cb373: Log is configured to *not* fsync() on all Append() calls
I20260811 06:30:29.041989 10051 tablet_bootstrap.cc:492] T 99a3f5ed1bea49f1aef444fbd19f32d8 P 34c9ac0ce67d4df5863681360f27c4ac: Bootstrap starting.
I20260811 06:30:29.045248 10049 tablet_bootstrap.cc:492] T 99a3f5ed1bea49f1aef444fbd19f32d8 P 2e276abb6ae14f2aab047be5cb9cb373: No bootstrap required, opened a new log
I20260811 06:30:29.045619 10049 ts_tablet_manager.cc:1397] T 99a3f5ed1bea49f1aef444fbd19f32d8 P 2e276abb6ae14f2aab047be5cb9cb373: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260811 06:30:29.046105 10051 tablet_bootstrap.cc:654] T 99a3f5ed1bea49f1aef444fbd19f32d8 P 34c9ac0ce67d4df5863681360f27c4ac: Neither blocks nor log segments found. Creating new log.
I20260811 06:30:29.046319 10050 tablet_bootstrap.cc:654] T 99a3f5ed1bea49f1aef444fbd19f32d8 P 57b2448bb01c461a964ee8ceca7f2315: Neither blocks nor log segments found. Creating new log.
I20260811 06:30:29.047497 10050 log.cc:824] T 99a3f5ed1bea49f1aef444fbd19f32d8 P 57b2448bb01c461a964ee8ceca7f2315: Log is configured to *not* fsync() on all Append() calls
I20260811 06:30:29.047497 10051 log.cc:824] T 99a3f5ed1bea49f1aef444fbd19f32d8 P 34c9ac0ce67d4df5863681360f27c4ac: Log is configured to *not* fsync() on all Append() calls
I20260811 06:30:29.053381 10050 tablet_bootstrap.cc:492] T 99a3f5ed1bea49f1aef444fbd19f32d8 P 57b2448bb01c461a964ee8ceca7f2315: No bootstrap required, opened a new log
I20260811 06:30:29.053793 10050 ts_tablet_manager.cc:1397] T 99a3f5ed1bea49f1aef444fbd19f32d8 P 57b2448bb01c461a964ee8ceca7f2315: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.006s
I20260811 06:30:29.053898 10051 tablet_bootstrap.cc:492] T 99a3f5ed1bea49f1aef444fbd19f32d8 P 34c9ac0ce67d4df5863681360f27c4ac: No bootstrap required, opened a new log
I20260811 06:30:29.054179 10051 ts_tablet_manager.cc:1397] T 99a3f5ed1bea49f1aef444fbd19f32d8 P 34c9ac0ce67d4df5863681360f27c4ac: Time spent bootstrapping tablet: real 0.012s user 0.008s sys 0.003s
I20260811 06:30:29.055265 10049 raft_consensus.cc:348] T 99a3f5ed1bea49f1aef444fbd19f32d8 P 2e276abb6ae14f2aab047be5cb9cb373 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b2448bb01c461a964ee8ceca7f2315" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 42829 } } peers { permanent_uuid: "34c9ac0ce67d4df5863681360f27c4ac" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 37827 } } peers { permanent_uuid: "2e276abb6ae14f2aab047be5cb9cb373" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37681 } }
I20260811 06:30:29.055866 10049 raft_consensus.cc:374] T 99a3f5ed1bea49f1aef444fbd19f32d8 P 2e276abb6ae14f2aab047be5cb9cb373 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:30:29.056017 10049 raft_consensus.cc:729] T 99a3f5ed1bea49f1aef444fbd19f32d8 P 2e276abb6ae14f2aab047be5cb9cb373 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e276abb6ae14f2aab047be5cb9cb373, State: Initialized, Role: FOLLOWER
I20260811 06:30:29.056820 10049 consensus_queue.cc:260] T 99a3f5ed1bea49f1aef444fbd19f32d8 P 2e276abb6ae14f2aab047be5cb9cb373 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b2448bb01c461a964ee8ceca7f2315" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 42829 } } peers { permanent_uuid: "34c9ac0ce67d4df5863681360f27c4ac" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 37827 } } peers { permanent_uuid: "2e276abb6ae14f2aab047be5cb9cb373" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37681 } }
I20260811 06:30:29.058075 9812 heartbeater.cc:503] Master 127.28.239.254:46215 was elected leader, sending a full tablet report...
I20260811 06:30:29.060672 10049 ts_tablet_manager.cc:1428] T 99a3f5ed1bea49f1aef444fbd19f32d8 P 2e276abb6ae14f2aab047be5cb9cb373: Time spent starting tablet: real 0.015s user 0.009s sys 0.003s
I20260811 06:30:29.062877 10050 raft_consensus.cc:348] T 99a3f5ed1bea49f1aef444fbd19f32d8 P 57b2448bb01c461a964ee8ceca7f2315 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b2448bb01c461a964ee8ceca7f2315" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 42829 } } peers { permanent_uuid: "34c9ac0ce67d4df5863681360f27c4ac" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 37827 } } peers { permanent_uuid: "2e276abb6ae14f2aab047be5cb9cb373" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37681 } }
I20260811 06:30:29.063258 10050 raft_consensus.cc:374] T 99a3f5ed1bea49f1aef444fbd19f32d8 P 57b2448bb01c461a964ee8ceca7f2315 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:30:29.063388 10050 raft_consensus.cc:729] T 99a3f5ed1bea49f1aef444fbd19f32d8 P 57b2448bb01c461a964ee8ceca7f2315 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57b2448bb01c461a964ee8ceca7f2315, State: Initialized, Role: FOLLOWER
I20260811 06:30:29.064031 10050 consensus_queue.cc:260] T 99a3f5ed1bea49f1aef444fbd19f32d8 P 57b2448bb01c461a964ee8ceca7f2315 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b2448bb01c461a964ee8ceca7f2315" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 42829 } } peers { permanent_uuid: "34c9ac0ce67d4df5863681360f27c4ac" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 37827 } } peers { permanent_uuid: "2e276abb6ae14f2aab047be5cb9cb373" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37681 } }
I20260811 06:30:29.064536 10051 raft_consensus.cc:348] T 99a3f5ed1bea49f1aef444fbd19f32d8 P 34c9ac0ce67d4df5863681360f27c4ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b2448bb01c461a964ee8ceca7f2315" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 42829 } } peers { permanent_uuid: "34c9ac0ce67d4df5863681360f27c4ac" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 37827 } } peers { permanent_uuid: "2e276abb6ae14f2aab047be5cb9cb373" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37681 } }
I20260811 06:30:29.065120 10051 raft_consensus.cc:374] T 99a3f5ed1bea49f1aef444fbd19f32d8 P 34c9ac0ce67d4df5863681360f27c4ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:30:29.065267 10051 raft_consensus.cc:729] T 99a3f5ed1bea49f1aef444fbd19f32d8 P 34c9ac0ce67d4df5863681360f27c4ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34c9ac0ce67d4df5863681360f27c4ac, State: Initialized, Role: FOLLOWER
I20260811 06:30:29.065989 10050 ts_tablet_manager.cc:1428] T 99a3f5ed1bea49f1aef444fbd19f32d8 P 57b2448bb01c461a964ee8ceca7f2315: Time spent starting tablet: real 0.012s user 0.009s sys 0.003s
I20260811 06:30:29.065925 10051 consensus_queue.cc:260] T 99a3f5ed1bea49f1aef444fbd19f32d8 P 34c9ac0ce67d4df5863681360f27c4ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b2448bb01c461a964ee8ceca7f2315" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 42829 } } peers { permanent_uuid: "34c9ac0ce67d4df5863681360f27c4ac" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 37827 } } peers { permanent_uuid: "2e276abb6ae14f2aab047be5cb9cb373" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37681 } }
I20260811 06:30:29.067106 10013 heartbeater.cc:503] Master 127.28.239.254:46215 was elected leader, sending a full tablet report...
I20260811 06:30:29.067853 10051 ts_tablet_manager.cc:1428] T 99a3f5ed1bea49f1aef444fbd19f32d8 P 34c9ac0ce67d4df5863681360f27c4ac: Time spent starting tablet: real 0.014s user 0.009s sys 0.004s
I20260811 06:30:29.069674 10061 raft_consensus.cc:482] T 99a3f5ed1bea49f1aef444fbd19f32d8 P 34c9ac0ce67d4df5863681360f27c4ac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:30:29.069988 10061 raft_consensus.cc:504] T 99a3f5ed1bea49f1aef444fbd19f32d8 P 34c9ac0ce67d4df5863681360f27c4ac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b2448bb01c461a964ee8ceca7f2315" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 42829 } } peers { permanent_uuid: "34c9ac0ce67d4df5863681360f27c4ac" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 37827 } } peers { permanent_uuid: "2e276abb6ae14f2aab047be5cb9cb373" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37681 } }
I20260811 06:30:29.072037 10061 leader_election.cc:288] T 99a3f5ed1bea49f1aef444fbd19f32d8 P 34c9ac0ce67d4df5863681360f27c4ac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57b2448bb01c461a964ee8ceca7f2315 (127.28.239.193:42829), 2e276abb6ae14f2aab047be5cb9cb373 (127.28.239.194:37681)
W20260811 06:30:29.075644 9813 tablet.cc:2369] T 99a3f5ed1bea49f1aef444fbd19f32d8 P 2e276abb6ae14f2aab047be5cb9cb373: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:30:29.078835 10045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:29.072964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.195:36905 (local address 127.28.239.193:42829)
0811 06:30:29.073153 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:29.073158 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:29.073187 (+ 29us) server_negotiation.cc:407] Connection header received
0811 06:30:29.073252 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:29.073256 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:29.073344 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:29.073437 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:30:29.073972 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:29.074676 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:29.077003 (+ 2327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:29.077919 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:29.078101 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:29.078189 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:29.078470 (+ 281us) server_negotiation.cc:299] Negotiation successful
0811 06:30:29.078596 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 06:30:29.078866 9854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:29.072193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.193:42829 (local address 127.28.239.195:36905)
0811 06:30:29.072407 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:29.072429 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:29.072550 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:29.073668 (+ 1118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:29.073673 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:29.073700 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:30:29.073846 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:29.073853 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:29.074813 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:29.074818 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:29.076839 (+ 2021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:29.076853 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:29.078085 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:29.078091 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:29.078299 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:29.078324 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 06:30:29.078434 (+ 110us) client_negotiation.cc:240] Negotiation successful
0811 06:30:29.078570 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46,"mutex_wait_us":28}
I20260811 06:30:29.079666 10048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:29.072688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.195:40261 (local address 127.28.239.194:37681)
0811 06:30:29.072904 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:29.072909 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:29.073264 (+ 355us) server_negotiation.cc:407] Connection header received
0811 06:30:29.073465 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:29.073470 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:29.073980 (+ 510us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:29.074106 (+ 126us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:30:29.074618 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:29.075849 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:29.077936 (+ 2087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:29.078742 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:29.078938 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:29.079009 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:29.079346 (+ 337us) server_negotiation.cc:299] Negotiation successful
0811 06:30:29.079478 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 06:30:29.079856 10064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:29.072503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.194:37681 (local address 127.28.239.195:40261)
0811 06:30:29.073148 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:29.073167 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:29.073320 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:29.074120 (+ 800us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:29.074124 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:29.074146 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:30:29.074445 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:29.074453 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:29.076017 (+ 1564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:29.076022 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:29.077778 (+ 1756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:29.077790 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:29.078966 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:29.078972 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:29.079168 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:29.079201 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 06:30:29.079294 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 06:30:29.079453 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":496,"mutex_wait_us":1,"thread_start_us":494,"threads_started":1}
I20260811 06:30:29.080297 9531 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99a3f5ed1bea49f1aef444fbd19f32d8" candidate_uuid: "34c9ac0ce67d4df5863681360f27c4ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57b2448bb01c461a964ee8ceca7f2315" is_pre_election: true
I20260811 06:30:29.080799 9531 raft_consensus.cc:2454] T 99a3f5ed1bea49f1aef444fbd19f32d8 P 57b2448bb01c461a964ee8ceca7f2315 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34c9ac0ce67d4df5863681360f27c4ac in term 0.
I20260811 06:30:29.080845 9746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99a3f5ed1bea49f1aef444fbd19f32d8" candidate_uuid: "34c9ac0ce67d4df5863681360f27c4ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e276abb6ae14f2aab047be5cb9cb373" is_pre_election: true
I20260811 06:30:29.081333 9746 raft_consensus.cc:2454] T 99a3f5ed1bea49f1aef444fbd19f32d8 P 2e276abb6ae14f2aab047be5cb9cb373 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34c9ac0ce67d4df5863681360f27c4ac in term 0.
I20260811 06:30:29.081501 9845 leader_election.cc:302] T 99a3f5ed1bea49f1aef444fbd19f32d8 P 34c9ac0ce67d4df5863681360f27c4ac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34c9ac0ce67d4df5863681360f27c4ac, 57b2448bb01c461a964ee8ceca7f2315; no voters:
I20260811 06:30:29.081990 10061 raft_consensus.cc:2781] T 99a3f5ed1bea49f1aef444fbd19f32d8 P 34c9ac0ce67d4df5863681360f27c4ac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:30:29.082157 10061 raft_consensus.cc:482] T 99a3f5ed1bea49f1aef444fbd19f32d8 P 34c9ac0ce67d4df5863681360f27c4ac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:30:29.082298 10061 raft_consensus.cc:3037] T 99a3f5ed1bea49f1aef444fbd19f32d8 P 34c9ac0ce67d4df5863681360f27c4ac [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:30:29.084532 10061 raft_consensus.cc:504] T 99a3f5ed1bea49f1aef444fbd19f32d8 P 34c9ac0ce67d4df5863681360f27c4ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b2448bb01c461a964ee8ceca7f2315" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 42829 } } peers { permanent_uuid: "34c9ac0ce67d4df5863681360f27c4ac" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 37827 } } peers { permanent_uuid: "2e276abb6ae14f2aab047be5cb9cb373" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37681 } }
I20260811 06:30:29.085108 10061 leader_election.cc:288] T 99a3f5ed1bea49f1aef444fbd19f32d8 P 34c9ac0ce67d4df5863681360f27c4ac [CANDIDATE]: Term 1 election: Requested vote from peers 57b2448bb01c461a964ee8ceca7f2315 (127.28.239.193:42829), 2e276abb6ae14f2aab047be5cb9cb373 (127.28.239.194:37681)
I20260811 06:30:29.085609 9531 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99a3f5ed1bea49f1aef444fbd19f32d8" candidate_uuid: "34c9ac0ce67d4df5863681360f27c4ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57b2448bb01c461a964ee8ceca7f2315"
I20260811 06:30:29.085752 9746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99a3f5ed1bea49f1aef444fbd19f32d8" candidate_uuid: "34c9ac0ce67d4df5863681360f27c4ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e276abb6ae14f2aab047be5cb9cb373"
I20260811 06:30:29.085829 9531 raft_consensus.cc:3037] T 99a3f5ed1bea49f1aef444fbd19f32d8 P 57b2448bb01c461a964ee8ceca7f2315 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:30:29.086001 9746 raft_consensus.cc:3037] T 99a3f5ed1bea49f1aef444fbd19f32d8 P 2e276abb6ae14f2aab047be5cb9cb373 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:30:29.088086 9531 raft_consensus.cc:2454] T 99a3f5ed1bea49f1aef444fbd19f32d8 P 57b2448bb01c461a964ee8ceca7f2315 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34c9ac0ce67d4df5863681360f27c4ac in term 1.
I20260811 06:30:29.088290 9746 raft_consensus.cc:2454] T 99a3f5ed1bea49f1aef444fbd19f32d8 P 2e276abb6ae14f2aab047be5cb9cb373 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34c9ac0ce67d4df5863681360f27c4ac in term 1.
I20260811 06:30:29.088546 9845 leader_election.cc:302] T 99a3f5ed1bea49f1aef444fbd19f32d8 P 34c9ac0ce67d4df5863681360f27c4ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34c9ac0ce67d4df5863681360f27c4ac, 57b2448bb01c461a964ee8ceca7f2315; no voters:
I20260811 06:30:29.088833 10061 raft_consensus.cc:2781] T 99a3f5ed1bea49f1aef444fbd19f32d8 P 34c9ac0ce67d4df5863681360f27c4ac [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:30:29.089280 10061 raft_consensus.cc:686] T 99a3f5ed1bea49f1aef444fbd19f32d8 P 34c9ac0ce67d4df5863681360f27c4ac [term 1 LEADER]: Becoming Leader. State: Replica: 34c9ac0ce67d4df5863681360f27c4ac, State: Running, Role: LEADER
I20260811 06:30:29.089847 10061 consensus_queue.cc:237] T 99a3f5ed1bea49f1aef444fbd19f32d8 P 34c9ac0ce67d4df5863681360f27c4ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b2448bb01c461a964ee8ceca7f2315" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 42829 } } peers { permanent_uuid: "34c9ac0ce67d4df5863681360f27c4ac" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 37827 } } peers { permanent_uuid: "2e276abb6ae14f2aab047be5cb9cb373" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37681 } }
I20260811 06:30:29.095062 9265 catalog_manager.cc:5495] T 99a3f5ed1bea49f1aef444fbd19f32d8 P 34c9ac0ce67d4df5863681360f27c4ac reported cstate change: term changed from 0 to 1, leader changed from <none> to 34c9ac0ce67d4df5863681360f27c4ac (127.28.239.195). New cstate: current_term: 1 leader_uuid: "34c9ac0ce67d4df5863681360f27c4ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b2448bb01c461a964ee8ceca7f2315" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 42829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34c9ac0ce67d4df5863681360f27c4ac" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 37827 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2e276abb6ae14f2aab047be5cb9cb373" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37681 } health_report { overall_health: UNKNOWN } } }
W20260811 06:30:29.130678 10014 tablet.cc:2369] T 99a3f5ed1bea49f1aef444fbd19f32d8 P 34c9ac0ce67d4df5863681360f27c4ac: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:30:29.131080 10046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:29.125313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42696 (local address 127.28.239.195:37827)
0811 06:30:29.125604 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:29.125617 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:29.125637 (+ 20us) server_negotiation.cc:407] Connection header received
0811 06:30:29.125829 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:29.125832 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:29.125917 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:29.126057 (+ 140us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:30:29.126468 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:29.127240 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:29.128581 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:29.128995 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:29.129110 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:29.129646 (+ 536us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:30:29.130273 (+ 627us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:30:29.130403 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:29.130571 (+ 168us) server_negotiation.cc:299] Negotiation successful
0811 06:30:29.130665 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 06:30:29.139814 9531 raft_consensus.cc:1261] T 99a3f5ed1bea49f1aef444fbd19f32d8 P 57b2448bb01c461a964ee8ceca7f2315 [term 1 FOLLOWER]: Refusing update from remote peer 34c9ac0ce67d4df5863681360f27c4ac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:30:29.139815 9746 raft_consensus.cc:1261] T 99a3f5ed1bea49f1aef444fbd19f32d8 P 2e276abb6ae14f2aab047be5cb9cb373 [term 1 FOLLOWER]: Refusing update from remote peer 34c9ac0ce67d4df5863681360f27c4ac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:30:29.140831 10065 consensus_queue.cc:1035] T 99a3f5ed1bea49f1aef444fbd19f32d8 P 34c9ac0ce67d4df5863681360f27c4ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "57b2448bb01c461a964ee8ceca7f2315" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 42829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:30:29.141239 10061 consensus_queue.cc:1035] T 99a3f5ed1bea49f1aef444fbd19f32d8 P 34c9ac0ce67d4df5863681360f27c4ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e276abb6ae14f2aab047be5cb9cb373" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:30:29.171631 9265 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33940:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:30:29.171922 9265 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33940:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:30:29.176386 9265 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2678652809584c6cb6c707b112de9a62: Sending DeleteTablet for 3 replicas of tablet 99a3f5ed1bea49f1aef444fbd19f32d8
I20260811 06:30:29.177742 9910 tablet_service.cc:1514] Processing DeleteTablet for tablet 99a3f5ed1bea49f1aef444fbd19f32d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:30:29 UTC) from {username='oryx', principal='oryx/127.28.239.254@KRBTEST.COM'} at 127.0.0.1:42686
I20260811 06:30:29.177770 9490 tablet_service.cc:1514] Processing DeleteTablet for tablet 99a3f5ed1bea49f1aef444fbd19f32d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:30:29 UTC) from {username='oryx', principal='oryx/127.28.239.254@KRBTEST.COM'} at 127.0.0.1:51296
I20260811 06:30:29.178828 10084 tablet_replica.cc:330] T 99a3f5ed1bea49f1aef444fbd19f32d8 P 34c9ac0ce67d4df5863681360f27c4ac: stopping tablet replica
I20260811 06:30:29.179319 10085 tablet_replica.cc:330] T 99a3f5ed1bea49f1aef444fbd19f32d8 P 57b2448bb01c461a964ee8ceca7f2315: stopping tablet replica
I20260811 06:30:29.179378 9706 tablet_service.cc:1514] Processing DeleteTablet for tablet 99a3f5ed1bea49f1aef444fbd19f32d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:30:29 UTC) from {username='oryx', principal='oryx/127.28.239.254@KRBTEST.COM'} at 127.0.0.1:42208
I20260811 06:30:29.179456 10084 raft_consensus.cc:2229] T 99a3f5ed1bea49f1aef444fbd19f32d8 P 34c9ac0ce67d4df5863681360f27c4ac [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:30:29.179790 10085 raft_consensus.cc:2229] T 99a3f5ed1bea49f1aef444fbd19f32d8 P 57b2448bb01c461a964ee8ceca7f2315 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:30:29.180305 10085 raft_consensus.cc:2258] T 99a3f5ed1bea49f1aef444fbd19f32d8 P 57b2448bb01c461a964ee8ceca7f2315 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:30:29.180363 10084 raft_consensus.cc:2258] T 99a3f5ed1bea49f1aef444fbd19f32d8 P 34c9ac0ce67d4df5863681360f27c4ac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:30:29.182264 9265 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33940:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 06:30:29.182646 10085 ts_tablet_manager.cc:1905] T 99a3f5ed1bea49f1aef444fbd19f32d8 P 57b2448bb01c461a964ee8ceca7f2315: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 06:30:29.182997 9265 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:30:29.183069 10084 ts_tablet_manager.cc:1905] T 99a3f5ed1bea49f1aef444fbd19f32d8 P 34c9ac0ce67d4df5863681360f27c4ac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:30:29.189430 10085 ts_tablet_manager.cc:1918] T 99a3f5ed1bea49f1aef444fbd19f32d8 P 57b2448bb01c461a964ee8ceca7f2315: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:30:29.189651 10085 log.cc:1196] T 99a3f5ed1bea49f1aef444fbd19f32d8 P 57b2448bb01c461a964ee8ceca7f2315: Deleting WAL directory at /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/minicluster-data/ts-0/wal/wals/99a3f5ed1bea49f1aef444fbd19f32d8
I20260811 06:30:29.190284 10085 ts_tablet_manager.cc:1939] T 99a3f5ed1bea49f1aef444fbd19f32d8 P 57b2448bb01c461a964ee8ceca7f2315: Deleting consensus metadata
I20260811 06:30:29.192698 9244 catalog_manager.cc:4841] TS 57b2448bb01c461a964ee8ceca7f2315 (127.28.239.193:42829): tablet 99a3f5ed1bea49f1aef444fbd19f32d8 (table test-table [id=49f84001a1f549e09dfb1390f4b741fe]) successfully deleted
I20260811 06:30:29.192728 9910 tablet_service.cc:1467] Processing CreateTablet for tablet 5a4e01e0edc747c7bd2fad848ed47824 (DEFAULT_TABLE table=test-table [id=0829373076a74d13bb2f80ffca798d12]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:30:29.193219 9910 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a4e01e0edc747c7bd2fad848ed47824. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:30:29.193868 9490 tablet_service.cc:1467] Processing CreateTablet for tablet 5a4e01e0edc747c7bd2fad848ed47824 (DEFAULT_TABLE table=test-table [id=0829373076a74d13bb2f80ffca798d12]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:30:29.194073 9705 tablet_service.cc:1467] Processing CreateTablet for tablet 5a4e01e0edc747c7bd2fad848ed47824 (DEFAULT_TABLE table=test-table [id=0829373076a74d13bb2f80ffca798d12]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:30:29.194320 9490 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a4e01e0edc747c7bd2fad848ed47824. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:30:29.194748 9705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a4e01e0edc747c7bd2fad848ed47824. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:30:29.195986 10086 tablet_replica.cc:330] T 99a3f5ed1bea49f1aef444fbd19f32d8 P 2e276abb6ae14f2aab047be5cb9cb373: stopping tablet replica
I20260811 06:30:29.196521 10086 raft_consensus.cc:2229] T 99a3f5ed1bea49f1aef444fbd19f32d8 P 2e276abb6ae14f2aab047be5cb9cb373 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:30:29.197249 10086 raft_consensus.cc:2258] T 99a3f5ed1bea49f1aef444fbd19f32d8 P 2e276abb6ae14f2aab047be5cb9cb373 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:30:29.199851 10086 ts_tablet_manager.cc:1905] T 99a3f5ed1bea49f1aef444fbd19f32d8 P 2e276abb6ae14f2aab047be5cb9cb373: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:30:29.200716 10050 tablet_bootstrap.cc:492] T 5a4e01e0edc747c7bd2fad848ed47824 P 57b2448bb01c461a964ee8ceca7f2315: Bootstrap starting.
I20260811 06:30:29.202453 10050 tablet_bootstrap.cc:654] T 5a4e01e0edc747c7bd2fad848ed47824 P 57b2448bb01c461a964ee8ceca7f2315: Neither blocks nor log segments found. Creating new log.
I20260811 06:30:29.206043 10050 tablet_bootstrap.cc:492] T 5a4e01e0edc747c7bd2fad848ed47824 P 57b2448bb01c461a964ee8ceca7f2315: No bootstrap required, opened a new log
I20260811 06:30:29.206211 10050 ts_tablet_manager.cc:1397] T 5a4e01e0edc747c7bd2fad848ed47824 P 57b2448bb01c461a964ee8ceca7f2315: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260811 06:30:29.207587 10084 ts_tablet_manager.cc:1918] T 99a3f5ed1bea49f1aef444fbd19f32d8 P 34c9ac0ce67d4df5863681360f27c4ac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:30:29.207978 10084 log.cc:1196] T 99a3f5ed1bea49f1aef444fbd19f32d8 P 34c9ac0ce67d4df5863681360f27c4ac: Deleting WAL directory at /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/minicluster-data/ts-2/wal/wals/99a3f5ed1bea49f1aef444fbd19f32d8
I20260811 06:30:29.208711 10084 ts_tablet_manager.cc:1939] T 99a3f5ed1bea49f1aef444fbd19f32d8 P 34c9ac0ce67d4df5863681360f27c4ac: Deleting consensus metadata
I20260811 06:30:29.208595 10050 raft_consensus.cc:348] T 5a4e01e0edc747c7bd2fad848ed47824 P 57b2448bb01c461a964ee8ceca7f2315 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b2448bb01c461a964ee8ceca7f2315" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 42829 } } peers { permanent_uuid: "2e276abb6ae14f2aab047be5cb9cb373" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37681 } } peers { permanent_uuid: "34c9ac0ce67d4df5863681360f27c4ac" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 37827 } }
I20260811 06:30:29.208832 10050 raft_consensus.cc:374] T 5a4e01e0edc747c7bd2fad848ed47824 P 57b2448bb01c461a964ee8ceca7f2315 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:30:29.208946 10050 raft_consensus.cc:729] T 5a4e01e0edc747c7bd2fad848ed47824 P 57b2448bb01c461a964ee8ceca7f2315 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57b2448bb01c461a964ee8ceca7f2315, State: Initialized, Role: FOLLOWER
I20260811 06:30:29.208997 10051 tablet_bootstrap.cc:492] T 5a4e01e0edc747c7bd2fad848ed47824 P 34c9ac0ce67d4df5863681360f27c4ac: Bootstrap starting.
I20260811 06:30:29.209237 10050 consensus_queue.cc:260] T 5a4e01e0edc747c7bd2fad848ed47824 P 57b2448bb01c461a964ee8ceca7f2315 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b2448bb01c461a964ee8ceca7f2315" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 42829 } } peers { permanent_uuid: "2e276abb6ae14f2aab047be5cb9cb373" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37681 } } peers { permanent_uuid: "34c9ac0ce67d4df5863681360f27c4ac" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 37827 } }
I20260811 06:30:29.210906 10051 tablet_bootstrap.cc:654] T 5a4e01e0edc747c7bd2fad848ed47824 P 34c9ac0ce67d4df5863681360f27c4ac: Neither blocks nor log segments found. Creating new log.
I20260811 06:30:29.211146 9242 catalog_manager.cc:4841] TS 34c9ac0ce67d4df5863681360f27c4ac (127.28.239.195:37827): tablet 99a3f5ed1bea49f1aef444fbd19f32d8 (table test-table [id=49f84001a1f549e09dfb1390f4b741fe]) successfully deleted
I20260811 06:30:29.212363 10049 tablet_bootstrap.cc:492] T 5a4e01e0edc747c7bd2fad848ed47824 P 2e276abb6ae14f2aab047be5cb9cb373: Bootstrap starting.
I20260811 06:30:29.213517 10050 ts_tablet_manager.cc:1428] T 5a4e01e0edc747c7bd2fad848ed47824 P 57b2448bb01c461a964ee8ceca7f2315: Time spent starting tablet: real 0.007s user 0.000s sys 0.003s
I20260811 06:30:29.213683 10051 tablet_bootstrap.cc:492] T 5a4e01e0edc747c7bd2fad848ed47824 P 34c9ac0ce67d4df5863681360f27c4ac: No bootstrap required, opened a new log
I20260811 06:30:29.213888 10051 ts_tablet_manager.cc:1397] T 5a4e01e0edc747c7bd2fad848ed47824 P 34c9ac0ce67d4df5863681360f27c4ac: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 06:30:29.214404 10049 tablet_bootstrap.cc:654] T 5a4e01e0edc747c7bd2fad848ed47824 P 2e276abb6ae14f2aab047be5cb9cb373: Neither blocks nor log segments found. Creating new log.
I20260811 06:30:29.214916 10051 raft_consensus.cc:348] T 5a4e01e0edc747c7bd2fad848ed47824 P 34c9ac0ce67d4df5863681360f27c4ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b2448bb01c461a964ee8ceca7f2315" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 42829 } } peers { permanent_uuid: "2e276abb6ae14f2aab047be5cb9cb373" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37681 } } peers { permanent_uuid: "34c9ac0ce67d4df5863681360f27c4ac" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 37827 } }
I20260811 06:30:29.215160 10051 raft_consensus.cc:374] T 5a4e01e0edc747c7bd2fad848ed47824 P 34c9ac0ce67d4df5863681360f27c4ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:30:29.215271 10051 raft_consensus.cc:729] T 5a4e01e0edc747c7bd2fad848ed47824 P 34c9ac0ce67d4df5863681360f27c4ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34c9ac0ce67d4df5863681360f27c4ac, State: Initialized, Role: FOLLOWER
I20260811 06:30:29.215613 10051 consensus_queue.cc:260] T 5a4e01e0edc747c7bd2fad848ed47824 P 34c9ac0ce67d4df5863681360f27c4ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b2448bb01c461a964ee8ceca7f2315" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 42829 } } peers { permanent_uuid: "2e276abb6ae14f2aab047be5cb9cb373" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37681 } } peers { permanent_uuid: "34c9ac0ce67d4df5863681360f27c4ac" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 37827 } }
I20260811 06:30:29.216732 10051 ts_tablet_manager.cc:1428] T 5a4e01e0edc747c7bd2fad848ed47824 P 34c9ac0ce67d4df5863681360f27c4ac: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260811 06:30:29.216773 10049 tablet_bootstrap.cc:492] T 5a4e01e0edc747c7bd2fad848ed47824 P 2e276abb6ae14f2aab047be5cb9cb373: No bootstrap required, opened a new log
I20260811 06:30:29.217038 10049 ts_tablet_manager.cc:1397] T 5a4e01e0edc747c7bd2fad848ed47824 P 2e276abb6ae14f2aab047be5cb9cb373: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.001s
I20260811 06:30:29.217989 10086 ts_tablet_manager.cc:1918] T 99a3f5ed1bea49f1aef444fbd19f32d8 P 2e276abb6ae14f2aab047be5cb9cb373: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:30:29.218200 10086 log.cc:1196] T 99a3f5ed1bea49f1aef444fbd19f32d8 P 2e276abb6ae14f2aab047be5cb9cb373: Deleting WAL directory at /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/minicluster-data/ts-1/wal/wals/99a3f5ed1bea49f1aef444fbd19f32d8
I20260811 06:30:29.218259 10049 raft_consensus.cc:348] T 5a4e01e0edc747c7bd2fad848ed47824 P 2e276abb6ae14f2aab047be5cb9cb373 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b2448bb01c461a964ee8ceca7f2315" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 42829 } } peers { permanent_uuid: "2e276abb6ae14f2aab047be5cb9cb373" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37681 } } peers { permanent_uuid: "34c9ac0ce67d4df5863681360f27c4ac" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 37827 } }
I20260811 06:30:29.218519 10049 raft_consensus.cc:374] T 5a4e01e0edc747c7bd2fad848ed47824 P 2e276abb6ae14f2aab047be5cb9cb373 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:30:29.218636 10049 raft_consensus.cc:729] T 5a4e01e0edc747c7bd2fad848ed47824 P 2e276abb6ae14f2aab047be5cb9cb373 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e276abb6ae14f2aab047be5cb9cb373, State: Initialized, Role: FOLLOWER
I20260811 06:30:29.219000 10086 ts_tablet_manager.cc:1939] T 99a3f5ed1bea49f1aef444fbd19f32d8 P 2e276abb6ae14f2aab047be5cb9cb373: Deleting consensus metadata
I20260811 06:30:29.218956 10049 consensus_queue.cc:260] T 5a4e01e0edc747c7bd2fad848ed47824 P 2e276abb6ae14f2aab047be5cb9cb373 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b2448bb01c461a964ee8ceca7f2315" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 42829 } } peers { permanent_uuid: "2e276abb6ae14f2aab047be5cb9cb373" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37681 } } peers { permanent_uuid: "34c9ac0ce67d4df5863681360f27c4ac" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 37827 } }
I20260811 06:30:29.219578 10049 ts_tablet_manager.cc:1428] T 5a4e01e0edc747c7bd2fad848ed47824 P 2e276abb6ae14f2aab047be5cb9cb373: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260811 06:30:29.221014 9244 catalog_manager.cc:4841] TS 2e276abb6ae14f2aab047be5cb9cb373 (127.28.239.194:37681): tablet 99a3f5ed1bea49f1aef444fbd19f32d8 (table test-table [id=49f84001a1f549e09dfb1390f4b741fe]) successfully deleted
W20260811 06:30:29.232559 9607 tablet.cc:2369] T 5a4e01e0edc747c7bd2fad848ed47824 P 57b2448bb01c461a964ee8ceca7f2315: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:30:29.242571 10060 raft_consensus.cc:482] T 5a4e01e0edc747c7bd2fad848ed47824 P 57b2448bb01c461a964ee8ceca7f2315 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:30:29.242820 10060 raft_consensus.cc:504] T 5a4e01e0edc747c7bd2fad848ed47824 P 57b2448bb01c461a964ee8ceca7f2315 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b2448bb01c461a964ee8ceca7f2315" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 42829 } } peers { permanent_uuid: "2e276abb6ae14f2aab047be5cb9cb373" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37681 } } peers { permanent_uuid: "34c9ac0ce67d4df5863681360f27c4ac" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 37827 } }
I20260811 06:30:29.244652 10060 leader_election.cc:288] T 5a4e01e0edc747c7bd2fad848ed47824 P 57b2448bb01c461a964ee8ceca7f2315 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2e276abb6ae14f2aab047be5cb9cb373 (127.28.239.194:37681), 34c9ac0ce67d4df5863681360f27c4ac (127.28.239.195:37827)
I20260811 06:30:29.253899 10048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:29.246016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.193:59873 (local address 127.28.239.194:37681)
0811 06:30:29.246256 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:29.246265 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:29.247057 (+ 792us) server_negotiation.cc:407] Connection header received
0811 06:30:29.247391 (+ 334us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:29.247397 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:29.247480 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:29.247589 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:30:29.248340 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:29.249512 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:29.251711 (+ 2199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:29.252744 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:29.252909 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:29.252997 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:29.253390 (+ 393us) server_negotiation.cc:299] Negotiation successful
0811 06:30:29.253577 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 06:30:29.254007 10091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:29.244836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.194:37681 (local address 127.28.239.193:59873)
0811 06:30:29.246926 (+ 2090us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:29.246960 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:29.247192 (+ 232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:29.247661 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:29.247680 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:29.247719 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:30:29.248128 (+ 409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:29.248147 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:29.249711 (+ 1564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:29.249716 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:29.251513 (+ 1797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:29.251527 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:29.252988 (+ 1461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:29.252995 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:29.253197 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:29.253234 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 06:30:29.253341 (+ 107us) client_negotiation.cc:240] Negotiation successful
0811 06:30:29.253483 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1897,"thread_start_us":1999,"threads_started":1}
I20260811 06:30:29.254714 10046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:29.245838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.193:44141 (local address 127.28.239.195:37827)
0811 06:30:29.246059 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:29.246067 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:29.247268 (+ 1201us) server_negotiation.cc:407] Connection header received
0811 06:30:29.247814 (+ 546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:29.247820 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:29.247901 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:29.248001 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:30:29.248643 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:29.249586 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:29.252235 (+ 2649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:29.253409 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:29.253574 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:29.253646 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:29.254298 (+ 652us) server_negotiation.cc:299] Negotiation successful
0811 06:30:29.254437 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 06:30:29.255115 9746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a4e01e0edc747c7bd2fad848ed47824" candidate_uuid: "57b2448bb01c461a964ee8ceca7f2315" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e276abb6ae14f2aab047be5cb9cb373" is_pre_election: true
I20260811 06:30:29.255439 10090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:29.245141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.195:37827 (local address 127.28.239.193:44141)
0811 06:30:29.247142 (+ 2001us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:29.247160 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:29.247274 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:29.248042 (+ 768us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:29.248048 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:29.248088 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:30:29.248487 (+ 399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:29.248496 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:29.249755 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:29.249761 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:29.252023 (+ 2262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:29.252039 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:29.253850 (+ 1811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:29.253856 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:29.254090 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:29.254120 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 06:30:29.254902 (+ 782us) client_negotiation.cc:240] Negotiation successful
0811 06:30:29.255033 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1890,"mutex_wait_us":2,"thread_start_us":1338,"threads_started":1}
I20260811 06:30:29.255389 9746 raft_consensus.cc:2454] T 5a4e01e0edc747c7bd2fad848ed47824 P 2e276abb6ae14f2aab047be5cb9cb373 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57b2448bb01c461a964ee8ceca7f2315 in term 0.
I20260811 06:30:29.256361 9939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a4e01e0edc747c7bd2fad848ed47824" candidate_uuid: "57b2448bb01c461a964ee8ceca7f2315" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34c9ac0ce67d4df5863681360f27c4ac" is_pre_election: true
I20260811 06:30:29.256698 9413 leader_election.cc:302] T 5a4e01e0edc747c7bd2fad848ed47824 P 57b2448bb01c461a964ee8ceca7f2315 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e276abb6ae14f2aab047be5cb9cb373, 57b2448bb01c461a964ee8ceca7f2315; no voters:
I20260811 06:30:29.256894 9939 raft_consensus.cc:2454] T 5a4e01e0edc747c7bd2fad848ed47824 P 34c9ac0ce67d4df5863681360f27c4ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57b2448bb01c461a964ee8ceca7f2315 in term 0.
I20260811 06:30:29.257170 10060 raft_consensus.cc:2781] T 5a4e01e0edc747c7bd2fad848ed47824 P 57b2448bb01c461a964ee8ceca7f2315 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:30:29.257300 10060 raft_consensus.cc:482] T 5a4e01e0edc747c7bd2fad848ed47824 P 57b2448bb01c461a964ee8ceca7f2315 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:30:29.257404 10060 raft_consensus.cc:3037] T 5a4e01e0edc747c7bd2fad848ed47824 P 57b2448bb01c461a964ee8ceca7f2315 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:30:29.260741 10060 raft_consensus.cc:504] T 5a4e01e0edc747c7bd2fad848ed47824 P 57b2448bb01c461a964ee8ceca7f2315 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b2448bb01c461a964ee8ceca7f2315" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 42829 } } peers { permanent_uuid: "2e276abb6ae14f2aab047be5cb9cb373" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37681 } } peers { permanent_uuid: "34c9ac0ce67d4df5863681360f27c4ac" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 37827 } }
I20260811 06:30:29.261557 10060 leader_election.cc:288] T 5a4e01e0edc747c7bd2fad848ed47824 P 57b2448bb01c461a964ee8ceca7f2315 [CANDIDATE]: Term 1 election: Requested vote from peers 2e276abb6ae14f2aab047be5cb9cb373 (127.28.239.194:37681), 34c9ac0ce67d4df5863681360f27c4ac (127.28.239.195:37827)
I20260811 06:30:29.262108 9939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a4e01e0edc747c7bd2fad848ed47824" candidate_uuid: "57b2448bb01c461a964ee8ceca7f2315" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34c9ac0ce67d4df5863681360f27c4ac"
I20260811 06:30:29.262140 9746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a4e01e0edc747c7bd2fad848ed47824" candidate_uuid: "57b2448bb01c461a964ee8ceca7f2315" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e276abb6ae14f2aab047be5cb9cb373"
I20260811 06:30:29.262360 9939 raft_consensus.cc:3037] T 5a4e01e0edc747c7bd2fad848ed47824 P 34c9ac0ce67d4df5863681360f27c4ac [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:30:29.262368 9746 raft_consensus.cc:3037] T 5a4e01e0edc747c7bd2fad848ed47824 P 2e276abb6ae14f2aab047be5cb9cb373 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:30:29.264976 9939 raft_consensus.cc:2454] T 5a4e01e0edc747c7bd2fad848ed47824 P 34c9ac0ce67d4df5863681360f27c4ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57b2448bb01c461a964ee8ceca7f2315 in term 1.
I20260811 06:30:29.265436 9746 raft_consensus.cc:2454] T 5a4e01e0edc747c7bd2fad848ed47824 P 2e276abb6ae14f2aab047be5cb9cb373 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57b2448bb01c461a964ee8ceca7f2315 in term 1.
I20260811 06:30:29.265470 9411 leader_election.cc:302] T 5a4e01e0edc747c7bd2fad848ed47824 P 57b2448bb01c461a964ee8ceca7f2315 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34c9ac0ce67d4df5863681360f27c4ac, 57b2448bb01c461a964ee8ceca7f2315; no voters:
I20260811 06:30:29.265883 10060 raft_consensus.cc:2781] T 5a4e01e0edc747c7bd2fad848ed47824 P 57b2448bb01c461a964ee8ceca7f2315 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:30:29.266443 10060 raft_consensus.cc:686] T 5a4e01e0edc747c7bd2fad848ed47824 P 57b2448bb01c461a964ee8ceca7f2315 [term 1 LEADER]: Becoming Leader. State: Replica: 57b2448bb01c461a964ee8ceca7f2315, State: Running, Role: LEADER
I20260811 06:30:29.267112 10060 consensus_queue.cc:237] T 5a4e01e0edc747c7bd2fad848ed47824 P 57b2448bb01c461a964ee8ceca7f2315 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b2448bb01c461a964ee8ceca7f2315" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 42829 } } peers { permanent_uuid: "2e276abb6ae14f2aab047be5cb9cb373" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37681 } } peers { permanent_uuid: "34c9ac0ce67d4df5863681360f27c4ac" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 37827 } }
I20260811 06:30:29.270761 9265 catalog_manager.cc:5495] T 5a4e01e0edc747c7bd2fad848ed47824 P 57b2448bb01c461a964ee8ceca7f2315 reported cstate change: term changed from 0 to 1, leader changed from <none> to 57b2448bb01c461a964ee8ceca7f2315 (127.28.239.193). New cstate: current_term: 1 leader_uuid: "57b2448bb01c461a964ee8ceca7f2315" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b2448bb01c461a964ee8ceca7f2315" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 42829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2e276abb6ae14f2aab047be5cb9cb373" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34c9ac0ce67d4df5863681360f27c4ac" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 37827 } health_report { overall_health: UNKNOWN } } }
Aug 11 06:30:29 dist-test-slave-gkw7 krb5kdc[9201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429826, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.239.254@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM
I20260811 06:30:29.334904 10109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:29.312388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:46215 (local address 127.0.0.1:33942)
0811 06:30:29.313122 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:29.313143 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:29.313301 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:29.313689 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:29.313694 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:29.314692 (+ 998us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:29.315032 (+ 340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:29.315056 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:29.316099 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:29.316105 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:29.317731 (+ 1626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:29.317747 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:29.318509 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:29.318517 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:29.318693 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:29.324099 (+ 5406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:30:29.324125 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:29.327483 (+ 3358us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:30:29.331642 (+ 4159us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:30:29.331658 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:30:29.331667 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:30:29.332348 (+ 681us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:30:29.332778 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:30:29.332784 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:30:29.332787 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:30:29.332982 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:30:29.333620 (+ 638us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:30:29.333630 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:30:29.333844 (+ 214us) client_negotiation.cc:769] Sending connection context
0811 06:30:29.334040 (+ 196us) client_negotiation.cc:240] Negotiation successful
0811 06:30:29.334401 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":552,"thread_start_us":383,"threads_started":1}
I20260811 06:30:29.334990 9852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:29.312389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33942 (local address 127.28.239.254:46215)
0811 06:30:29.312678 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:29.312687 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:29.313241 (+ 554us) server_negotiation.cc:407] Connection header received
0811 06:30:29.313424 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:29.313427 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:29.313522 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:29.313652 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:29.315229 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:29.315953 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:29.317878 (+ 1925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:29.318308 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:29.318406 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:29.327742 (+ 9336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:29.327749 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:29.327754 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:30:29.327795 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:29.331454 (+ 3659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:29.332475 (+ 1021us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:29.332482 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:29.332485 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:29.332637 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:29.333101 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:29.333106 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:29.333110 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:29.333455 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:29.333633 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:29.334147 (+ 514us) server_negotiation.cc:299] Negotiation successful
0811 06:30:29.334596 (+ 449us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":182,"server-negotiator.queue_time_us":129}
I20260811 06:30:29.341694 9265 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.28.239.254@KRBTEST.COM'} at 127.0.0.1:33942:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 06:30:29.343099 9265 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:30:29.351096 9705 tablet_service.cc:1467] Processing CreateTablet for tablet 27f2261a4ffd46b3a46599e1d0fcaa8b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b9bad9b45d5540ed8b36552318752081]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 06:30:29.351101 9490 tablet_service.cc:1467] Processing CreateTablet for tablet 27f2261a4ffd46b3a46599e1d0fcaa8b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b9bad9b45d5540ed8b36552318752081]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 06:30:29.351639 9705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 27f2261a4ffd46b3a46599e1d0fcaa8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:30:29.351666 9490 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 27f2261a4ffd46b3a46599e1d0fcaa8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:30:29.356271 9910 tablet_service.cc:1467] Processing CreateTablet for tablet 27f2261a4ffd46b3a46599e1d0fcaa8b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b9bad9b45d5540ed8b36552318752081]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 06:30:29.356945 9910 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 27f2261a4ffd46b3a46599e1d0fcaa8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:30:29.357363 10049 tablet_bootstrap.cc:492] T 27f2261a4ffd46b3a46599e1d0fcaa8b P 2e276abb6ae14f2aab047be5cb9cb373: Bootstrap starting.
I20260811 06:30:29.359030 10050 tablet_bootstrap.cc:492] T 27f2261a4ffd46b3a46599e1d0fcaa8b P 57b2448bb01c461a964ee8ceca7f2315: Bootstrap starting.
I20260811 06:30:29.359102 10049 tablet_bootstrap.cc:654] T 27f2261a4ffd46b3a46599e1d0fcaa8b P 2e276abb6ae14f2aab047be5cb9cb373: Neither blocks nor log segments found. Creating new log.
I20260811 06:30:29.360737 10050 tablet_bootstrap.cc:654] T 27f2261a4ffd46b3a46599e1d0fcaa8b P 57b2448bb01c461a964ee8ceca7f2315: Neither blocks nor log segments found. Creating new log.
I20260811 06:30:29.368359 10051 tablet_bootstrap.cc:492] T 27f2261a4ffd46b3a46599e1d0fcaa8b P 34c9ac0ce67d4df5863681360f27c4ac: Bootstrap starting.
I20260811 06:30:29.370657 10051 tablet_bootstrap.cc:654] T 27f2261a4ffd46b3a46599e1d0fcaa8b P 34c9ac0ce67d4df5863681360f27c4ac: Neither blocks nor log segments found. Creating new log.
I20260811 06:30:29.371047 10049 tablet_bootstrap.cc:492] T 27f2261a4ffd46b3a46599e1d0fcaa8b P 2e276abb6ae14f2aab047be5cb9cb373: No bootstrap required, opened a new log
I20260811 06:30:29.371207 10049 ts_tablet_manager.cc:1397] T 27f2261a4ffd46b3a46599e1d0fcaa8b P 2e276abb6ae14f2aab047be5cb9cb373: Time spent bootstrapping tablet: real 0.014s user 0.009s sys 0.003s
I20260811 06:30:29.372061 10049 raft_consensus.cc:348] T 27f2261a4ffd46b3a46599e1d0fcaa8b P 2e276abb6ae14f2aab047be5cb9cb373 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b2448bb01c461a964ee8ceca7f2315" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 42829 } } peers { permanent_uuid: "2e276abb6ae14f2aab047be5cb9cb373" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37681 } } peers { permanent_uuid: "34c9ac0ce67d4df5863681360f27c4ac" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 37827 } }
I20260811 06:30:29.372390 10049 raft_consensus.cc:374] T 27f2261a4ffd46b3a46599e1d0fcaa8b P 2e276abb6ae14f2aab047be5cb9cb373 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:30:29.372493 10049 raft_consensus.cc:729] T 27f2261a4ffd46b3a46599e1d0fcaa8b P 2e276abb6ae14f2aab047be5cb9cb373 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e276abb6ae14f2aab047be5cb9cb373, State: Initialized, Role: FOLLOWER
I20260811 06:30:29.372898 10049 consensus_queue.cc:260] T 27f2261a4ffd46b3a46599e1d0fcaa8b P 2e276abb6ae14f2aab047be5cb9cb373 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b2448bb01c461a964ee8ceca7f2315" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 42829 } } peers { permanent_uuid: "2e276abb6ae14f2aab047be5cb9cb373" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37681 } } peers { permanent_uuid: "34c9ac0ce67d4df5863681360f27c4ac" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 37827 } }
I20260811 06:30:29.373867 10049 ts_tablet_manager.cc:1428] T 27f2261a4ffd46b3a46599e1d0fcaa8b P 2e276abb6ae14f2aab047be5cb9cb373: Time spent starting tablet: real 0.002s user 0.003s sys 0.001s
I20260811 06:30:29.373513 10058 tablet_replica.cc:439] T 27f2261a4ffd46b3a46599e1d0fcaa8b P 2e276abb6ae14f2aab047be5cb9cb373: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b2448bb01c461a964ee8ceca7f2315" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 42829 } } peers { permanent_uuid: "2e276abb6ae14f2aab047be5cb9cb373" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37681 } } peers { permanent_uuid: "34c9ac0ce67d4df5863681360f27c4ac" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 37827 } } }
I20260811 06:30:29.374838 10058 tablet_replica.cc:442] T 27f2261a4ffd46b3a46599e1d0fcaa8b P 2e276abb6ae14f2aab047be5cb9cb373: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 06:30:29.379292 10050 tablet_bootstrap.cc:492] T 27f2261a4ffd46b3a46599e1d0fcaa8b P 57b2448bb01c461a964ee8ceca7f2315: No bootstrap required, opened a new log
I20260811 06:30:29.379504 10050 ts_tablet_manager.cc:1397] T 27f2261a4ffd46b3a46599e1d0fcaa8b P 57b2448bb01c461a964ee8ceca7f2315: Time spent bootstrapping tablet: real 0.021s user 0.004s sys 0.011s
I20260811 06:30:29.380331 10050 raft_consensus.cc:348] T 27f2261a4ffd46b3a46599e1d0fcaa8b P 57b2448bb01c461a964ee8ceca7f2315 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b2448bb01c461a964ee8ceca7f2315" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 42829 } } peers { permanent_uuid: "2e276abb6ae14f2aab047be5cb9cb373" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37681 } } peers { permanent_uuid: "34c9ac0ce67d4df5863681360f27c4ac" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 37827 } }
I20260811 06:30:29.380612 10050 raft_consensus.cc:374] T 27f2261a4ffd46b3a46599e1d0fcaa8b P 57b2448bb01c461a964ee8ceca7f2315 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:30:29.380699 10050 raft_consensus.cc:729] T 27f2261a4ffd46b3a46599e1d0fcaa8b P 57b2448bb01c461a964ee8ceca7f2315 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57b2448bb01c461a964ee8ceca7f2315, State: Initialized, Role: FOLLOWER
I20260811 06:30:29.380851 10051 tablet_bootstrap.cc:492] T 27f2261a4ffd46b3a46599e1d0fcaa8b P 34c9ac0ce67d4df5863681360f27c4ac: No bootstrap required, opened a new log
I20260811 06:30:29.381036 10051 ts_tablet_manager.cc:1397] T 27f2261a4ffd46b3a46599e1d0fcaa8b P 34c9ac0ce67d4df5863681360f27c4ac: Time spent bootstrapping tablet: real 0.013s user 0.006s sys 0.003s
I20260811 06:30:29.380955 10050 consensus_queue.cc:260] T 27f2261a4ffd46b3a46599e1d0fcaa8b P 57b2448bb01c461a964ee8ceca7f2315 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b2448bb01c461a964ee8ceca7f2315" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 42829 } } peers { permanent_uuid: "2e276abb6ae14f2aab047be5cb9cb373" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37681 } } peers { permanent_uuid: "34c9ac0ce67d4df5863681360f27c4ac" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 37827 } }
I20260811 06:30:29.381449 10060 tablet_replica.cc:439] T 27f2261a4ffd46b3a46599e1d0fcaa8b P 57b2448bb01c461a964ee8ceca7f2315: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b2448bb01c461a964ee8ceca7f2315" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 42829 } } peers { permanent_uuid: "2e276abb6ae14f2aab047be5cb9cb373" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37681 } } peers { permanent_uuid: "34c9ac0ce67d4df5863681360f27c4ac" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 37827 } } }
I20260811 06:30:29.381655 10050 ts_tablet_manager.cc:1428] T 27f2261a4ffd46b3a46599e1d0fcaa8b P 57b2448bb01c461a964ee8ceca7f2315: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 06:30:29.381853 10060 tablet_replica.cc:442] T 27f2261a4ffd46b3a46599e1d0fcaa8b P 57b2448bb01c461a964ee8ceca7f2315: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 06:30:29.382220 10051 raft_consensus.cc:348] T 27f2261a4ffd46b3a46599e1d0fcaa8b P 34c9ac0ce67d4df5863681360f27c4ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b2448bb01c461a964ee8ceca7f2315" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 42829 } } peers { permanent_uuid: "2e276abb6ae14f2aab047be5cb9cb373" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37681 } } peers { permanent_uuid: "34c9ac0ce67d4df5863681360f27c4ac" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 37827 } }
I20260811 06:30:29.382483 10051 raft_consensus.cc:374] T 27f2261a4ffd46b3a46599e1d0fcaa8b P 34c9ac0ce67d4df5863681360f27c4ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:30:29.382601 10051 raft_consensus.cc:729] T 27f2261a4ffd46b3a46599e1d0fcaa8b P 34c9ac0ce67d4df5863681360f27c4ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34c9ac0ce67d4df5863681360f27c4ac, State: Initialized, Role: FOLLOWER
I20260811 06:30:29.382891 10051 consensus_queue.cc:260] T 27f2261a4ffd46b3a46599e1d0fcaa8b P 34c9ac0ce67d4df5863681360f27c4ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b2448bb01c461a964ee8ceca7f2315" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 42829 } } peers { permanent_uuid: "2e276abb6ae14f2aab047be5cb9cb373" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37681 } } peers { permanent_uuid: "34c9ac0ce67d4df5863681360f27c4ac" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 37827 } }
I20260811 06:30:29.383508 10051 ts_tablet_manager.cc:1428] T 27f2261a4ffd46b3a46599e1d0fcaa8b P 34c9ac0ce67d4df5863681360f27c4ac: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 06:30:29.383378 10076 tablet_replica.cc:439] T 27f2261a4ffd46b3a46599e1d0fcaa8b P 34c9ac0ce67d4df5863681360f27c4ac: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b2448bb01c461a964ee8ceca7f2315" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 42829 } } peers { permanent_uuid: "2e276abb6ae14f2aab047be5cb9cb373" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37681 } } peers { permanent_uuid: "34c9ac0ce67d4df5863681360f27c4ac" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 37827 } } }
I20260811 06:30:29.383807 10076 tablet_replica.cc:442] T 27f2261a4ffd46b3a46599e1d0fcaa8b P 34c9ac0ce67d4df5863681360f27c4ac: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 06:30:29.384744 10060 raft_consensus.cc:482] T 27f2261a4ffd46b3a46599e1d0fcaa8b P 57b2448bb01c461a964ee8ceca7f2315 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:30:29.384912 10060 raft_consensus.cc:504] T 27f2261a4ffd46b3a46599e1d0fcaa8b P 57b2448bb01c461a964ee8ceca7f2315 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b2448bb01c461a964ee8ceca7f2315" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 42829 } } peers { permanent_uuid: "2e276abb6ae14f2aab047be5cb9cb373" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37681 } } peers { permanent_uuid: "34c9ac0ce67d4df5863681360f27c4ac" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 37827 } }
I20260811 06:30:29.385650 10060 leader_election.cc:288] T 27f2261a4ffd46b3a46599e1d0fcaa8b P 57b2448bb01c461a964ee8ceca7f2315 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2e276abb6ae14f2aab047be5cb9cb373 (127.28.239.194:37681), 34c9ac0ce67d4df5863681360f27c4ac (127.28.239.195:37827)
I20260811 06:30:29.386173 9939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27f2261a4ffd46b3a46599e1d0fcaa8b" candidate_uuid: "57b2448bb01c461a964ee8ceca7f2315" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34c9ac0ce67d4df5863681360f27c4ac" is_pre_election: true
I20260811 06:30:29.386166 9746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27f2261a4ffd46b3a46599e1d0fcaa8b" candidate_uuid: "57b2448bb01c461a964ee8ceca7f2315" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e276abb6ae14f2aab047be5cb9cb373" is_pre_election: true
I20260811 06:30:29.386420 9939 raft_consensus.cc:2454] T 27f2261a4ffd46b3a46599e1d0fcaa8b P 34c9ac0ce67d4df5863681360f27c4ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57b2448bb01c461a964ee8ceca7f2315 in term 0.
I20260811 06:30:29.386451 9746 raft_consensus.cc:2454] T 27f2261a4ffd46b3a46599e1d0fcaa8b P 2e276abb6ae14f2aab047be5cb9cb373 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57b2448bb01c461a964ee8ceca7f2315 in term 0.
I20260811 06:30:29.386885 9411 leader_election.cc:302] T 27f2261a4ffd46b3a46599e1d0fcaa8b P 57b2448bb01c461a964ee8ceca7f2315 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34c9ac0ce67d4df5863681360f27c4ac, 57b2448bb01c461a964ee8ceca7f2315; no voters:
I20260811 06:30:29.387315 10060 raft_consensus.cc:2781] T 27f2261a4ffd46b3a46599e1d0fcaa8b P 57b2448bb01c461a964ee8ceca7f2315 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:30:29.387439 10060 raft_consensus.cc:482] T 27f2261a4ffd46b3a46599e1d0fcaa8b P 57b2448bb01c461a964ee8ceca7f2315 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:30:29.387562 10060 raft_consensus.cc:3037] T 27f2261a4ffd46b3a46599e1d0fcaa8b P 57b2448bb01c461a964ee8ceca7f2315 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:30:29.390623 10060 raft_consensus.cc:504] T 27f2261a4ffd46b3a46599e1d0fcaa8b P 57b2448bb01c461a964ee8ceca7f2315 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b2448bb01c461a964ee8ceca7f2315" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 42829 } } peers { permanent_uuid: "2e276abb6ae14f2aab047be5cb9cb373" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37681 } } peers { permanent_uuid: "34c9ac0ce67d4df5863681360f27c4ac" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 37827 } }
I20260811 06:30:29.391729 9746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27f2261a4ffd46b3a46599e1d0fcaa8b" candidate_uuid: "57b2448bb01c461a964ee8ceca7f2315" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e276abb6ae14f2aab047be5cb9cb373"
I20260811 06:30:29.391922 9939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27f2261a4ffd46b3a46599e1d0fcaa8b" candidate_uuid: "57b2448bb01c461a964ee8ceca7f2315" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34c9ac0ce67d4df5863681360f27c4ac"
I20260811 06:30:29.392002 9746 raft_consensus.cc:3037] T 27f2261a4ffd46b3a46599e1d0fcaa8b P 2e276abb6ae14f2aab047be5cb9cb373 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:30:29.392155 9939 raft_consensus.cc:3037] T 27f2261a4ffd46b3a46599e1d0fcaa8b P 34c9ac0ce67d4df5863681360f27c4ac [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:30:29.394554 9746 raft_consensus.cc:2454] T 27f2261a4ffd46b3a46599e1d0fcaa8b P 2e276abb6ae14f2aab047be5cb9cb373 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57b2448bb01c461a964ee8ceca7f2315 in term 1.
I20260811 06:30:29.394953 9413 leader_election.cc:302] T 27f2261a4ffd46b3a46599e1d0fcaa8b P 57b2448bb01c461a964ee8ceca7f2315 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e276abb6ae14f2aab047be5cb9cb373, 57b2448bb01c461a964ee8ceca7f2315; no voters:
I20260811 06:30:29.395318 10093 raft_consensus.cc:2781] T 27f2261a4ffd46b3a46599e1d0fcaa8b P 57b2448bb01c461a964ee8ceca7f2315 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:30:29.395354 9939 raft_consensus.cc:2454] T 27f2261a4ffd46b3a46599e1d0fcaa8b P 34c9ac0ce67d4df5863681360f27c4ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57b2448bb01c461a964ee8ceca7f2315 in term 1.
I20260811 06:30:29.396404 10060 leader_election.cc:288] T 27f2261a4ffd46b3a46599e1d0fcaa8b P 57b2448bb01c461a964ee8ceca7f2315 [CANDIDATE]: Term 1 election: Requested vote from peers 2e276abb6ae14f2aab047be5cb9cb373 (127.28.239.194:37681), 34c9ac0ce67d4df5863681360f27c4ac (127.28.239.195:37827)
I20260811 06:30:29.396505 10093 raft_consensus.cc:686] T 27f2261a4ffd46b3a46599e1d0fcaa8b P 57b2448bb01c461a964ee8ceca7f2315 [term 1 LEADER]: Becoming Leader. State: Replica: 57b2448bb01c461a964ee8ceca7f2315, State: Running, Role: LEADER
I20260811 06:30:29.396945 10093 consensus_queue.cc:237] T 27f2261a4ffd46b3a46599e1d0fcaa8b P 57b2448bb01c461a964ee8ceca7f2315 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b2448bb01c461a964ee8ceca7f2315" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 42829 } } peers { permanent_uuid: "2e276abb6ae14f2aab047be5cb9cb373" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37681 } } peers { permanent_uuid: "34c9ac0ce67d4df5863681360f27c4ac" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 37827 } }
I20260811 06:30:29.399785 10115 tablet_replica.cc:439] T 27f2261a4ffd46b3a46599e1d0fcaa8b P 57b2448bb01c461a964ee8ceca7f2315: TxnStatusTablet state changed. Reason: New leader 57b2448bb01c461a964ee8ceca7f2315. Latest consensus state: current_term: 1 leader_uuid: "57b2448bb01c461a964ee8ceca7f2315" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b2448bb01c461a964ee8ceca7f2315" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 42829 } } peers { permanent_uuid: "2e276abb6ae14f2aab047be5cb9cb373" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37681 } } peers { permanent_uuid: "34c9ac0ce67d4df5863681360f27c4ac" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 37827 } } }
I20260811 06:30:29.400123 10115 tablet_replica.cc:442] T 27f2261a4ffd46b3a46599e1d0fcaa8b P 57b2448bb01c461a964ee8ceca7f2315: This TxnStatusTablet replica's current role is: LEADER
I20260811 06:30:29.400867 10117 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 06:30:29.400574 9265 catalog_manager.cc:5495] T 27f2261a4ffd46b3a46599e1d0fcaa8b P 57b2448bb01c461a964ee8ceca7f2315 reported cstate change: term changed from 0 to 1, leader changed from <none> to 57b2448bb01c461a964ee8ceca7f2315 (127.28.239.193). New cstate: current_term: 1 leader_uuid: "57b2448bb01c461a964ee8ceca7f2315" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b2448bb01c461a964ee8ceca7f2315" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 42829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2e276abb6ae14f2aab047be5cb9cb373" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34c9ac0ce67d4df5863681360f27c4ac" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 37827 } health_report { overall_health: UNKNOWN } } }
I20260811 06:30:29.401111 10117 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 06:30:29.418412 10045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:29.412196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51302 (local address 127.28.239.193:42829)
0811 06:30:29.412406 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:29.412412 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:29.412638 (+ 226us) server_negotiation.cc:407] Connection header received
0811 06:30:29.412940 (+ 302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:29.412945 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:29.413046 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:29.413225 (+ 179us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:30:29.413609 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:29.414709 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:29.416040 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:29.416501 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:29.416643 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:29.417138 (+ 495us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:30:29.417710 (+ 572us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:30:29.417876 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:29.417972 (+ 96us) server_negotiation.cc:299] Negotiation successful
0811 06:30:29.418052 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 06:30:29.418426 10109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:29.412335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.193:42829 (local address 127.0.0.1:51302)
0811 06:30:29.412542 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:29.412557 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:29.412729 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:29.413245 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:29.413249 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:29.413261 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 06:30:29.413452 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:29.413461 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:29.414863 (+ 1402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:29.414868 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:29.415892 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:29.415905 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:29.416672 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:29.416677 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:29.416882 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:29.416935 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 06:30:29.417833 (+ 898us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 06:30:29.417866 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 06:30:29.417954 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 06:30:29.418101 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260811 06:30:29.422974 9746 raft_consensus.cc:1261] T 27f2261a4ffd46b3a46599e1d0fcaa8b P 2e276abb6ae14f2aab047be5cb9cb373 [term 1 FOLLOWER]: Refusing update from remote peer 57b2448bb01c461a964ee8ceca7f2315: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:30:29.423094 9939 raft_consensus.cc:1261] T 27f2261a4ffd46b3a46599e1d0fcaa8b P 34c9ac0ce67d4df5863681360f27c4ac [term 1 FOLLOWER]: Refusing update from remote peer 57b2448bb01c461a964ee8ceca7f2315: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:30:29.423929 10093 consensus_queue.cc:1035] T 27f2261a4ffd46b3a46599e1d0fcaa8b P 57b2448bb01c461a964ee8ceca7f2315 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e276abb6ae14f2aab047be5cb9cb373" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:30:29.424386 10060 consensus_queue.cc:1035] T 27f2261a4ffd46b3a46599e1d0fcaa8b P 57b2448bb01c461a964ee8ceca7f2315 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34c9ac0ce67d4df5863681360f27c4ac" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 37827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:30:29.434535 10058 tablet_replica.cc:439] T 27f2261a4ffd46b3a46599e1d0fcaa8b P 2e276abb6ae14f2aab047be5cb9cb373: TxnStatusTablet state changed. Reason: New leader 57b2448bb01c461a964ee8ceca7f2315. Latest consensus state: current_term: 1 leader_uuid: "57b2448bb01c461a964ee8ceca7f2315" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b2448bb01c461a964ee8ceca7f2315" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 42829 } } peers { permanent_uuid: "2e276abb6ae14f2aab047be5cb9cb373" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37681 } } peers { permanent_uuid: "34c9ac0ce67d4df5863681360f27c4ac" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 37827 } } }
I20260811 06:30:29.434867 10058 tablet_replica.cc:442] T 27f2261a4ffd46b3a46599e1d0fcaa8b P 2e276abb6ae14f2aab047be5cb9cb373: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 06:30:29.435667 10076 tablet_replica.cc:439] T 27f2261a4ffd46b3a46599e1d0fcaa8b P 34c9ac0ce67d4df5863681360f27c4ac: TxnStatusTablet state changed. Reason: New leader 57b2448bb01c461a964ee8ceca7f2315. Latest consensus state: current_term: 1 leader_uuid: "57b2448bb01c461a964ee8ceca7f2315" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b2448bb01c461a964ee8ceca7f2315" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 42829 } } peers { permanent_uuid: "2e276abb6ae14f2aab047be5cb9cb373" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37681 } } peers { permanent_uuid: "34c9ac0ce67d4df5863681360f27c4ac" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 37827 } } }
I20260811 06:30:29.435948 10076 tablet_replica.cc:442] T 27f2261a4ffd46b3a46599e1d0fcaa8b P 34c9ac0ce67d4df5863681360f27c4ac: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 06:30:29.438244 10093 tablet_replica.cc:439] T 27f2261a4ffd46b3a46599e1d0fcaa8b P 57b2448bb01c461a964ee8ceca7f2315: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "57b2448bb01c461a964ee8ceca7f2315" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b2448bb01c461a964ee8ceca7f2315" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 42829 } } peers { permanent_uuid: "2e276abb6ae14f2aab047be5cb9cb373" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37681 } } peers { permanent_uuid: "34c9ac0ce67d4df5863681360f27c4ac" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 37827 } } }
I20260811 06:30:29.438536 10093 tablet_replica.cc:442] T 27f2261a4ffd46b3a46599e1d0fcaa8b P 57b2448bb01c461a964ee8ceca7f2315: This TxnStatusTablet replica's current role is: LEADER
I20260811 06:30:29.439669 10070 mvcc.cc:205] Tried to move back new op lower bound from 7317216581308157952 to 7317216581213171712. Current Snapshot: MvccSnapshot[applied={T|T < 7317216581308157952 or (T in {7317216581308157952})}]
I20260811 06:30:29.442725 10058 tablet_replica.cc:439] T 27f2261a4ffd46b3a46599e1d0fcaa8b P 2e276abb6ae14f2aab047be5cb9cb373: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "57b2448bb01c461a964ee8ceca7f2315" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b2448bb01c461a964ee8ceca7f2315" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 42829 } } peers { permanent_uuid: "2e276abb6ae14f2aab047be5cb9cb373" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37681 } } peers { permanent_uuid: "34c9ac0ce67d4df5863681360f27c4ac" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 37827 } } }
I20260811 06:30:29.442790 10060 tablet_replica.cc:439] T 27f2261a4ffd46b3a46599e1d0fcaa8b P 57b2448bb01c461a964ee8ceca7f2315: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "57b2448bb01c461a964ee8ceca7f2315" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b2448bb01c461a964ee8ceca7f2315" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 42829 } } peers { permanent_uuid: "2e276abb6ae14f2aab047be5cb9cb373" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37681 } } peers { permanent_uuid: "34c9ac0ce67d4df5863681360f27c4ac" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 37827 } } }
I20260811 06:30:29.442978 10058 tablet_replica.cc:442] T 27f2261a4ffd46b3a46599e1d0fcaa8b P 2e276abb6ae14f2aab047be5cb9cb373: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 06:30:29.443044 10060 tablet_replica.cc:442] T 27f2261a4ffd46b3a46599e1d0fcaa8b P 57b2448bb01c461a964ee8ceca7f2315: This TxnStatusTablet replica's current role is: LEADER
I20260811 06:30:29.447292 10076 tablet_replica.cc:439] T 27f2261a4ffd46b3a46599e1d0fcaa8b P 34c9ac0ce67d4df5863681360f27c4ac: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "57b2448bb01c461a964ee8ceca7f2315" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b2448bb01c461a964ee8ceca7f2315" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 42829 } } peers { permanent_uuid: "2e276abb6ae14f2aab047be5cb9cb373" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37681 } } peers { permanent_uuid: "34c9ac0ce67d4df5863681360f27c4ac" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 37827 } } }
I20260811 06:30:29.447569 10076 tablet_replica.cc:442] T 27f2261a4ffd46b3a46599e1d0fcaa8b P 34c9ac0ce67d4df5863681360f27c4ac: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 06:30:29.463865 10045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:29.454545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51318 (local address 127.28.239.193:42829)
0811 06:30:29.456014 (+ 1469us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:29.456023 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:29.456042 (+ 19us) server_negotiation.cc:407] Connection header received
0811 06:30:29.456137 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:29.456169 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:29.456296 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:29.457304 (+ 1008us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:30:29.457566 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:29.459070 (+ 1504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:29.461164 (+ 2094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:29.461907 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:29.462586 (+ 679us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:29.462713 (+ 127us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:30:29.462848 (+ 135us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:30:29.463396 (+ 548us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:29.463454 (+ 58us) server_negotiation.cc:299] Negotiation successful
0811 06:30:29.463533 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1210}
I20260811 06:30:29.484407 10045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:29.476691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.193:35439 (local address 127.28.239.193:42829)
0811 06:30:29.476863 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:29.476870 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:29.477487 (+ 617us) server_negotiation.cc:407] Connection header received
0811 06:30:29.477740 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:29.477746 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:29.477826 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:29.477920 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:30:29.478593 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:29.479788 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:29.482093 (+ 2305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:29.482802 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:29.482930 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:29.483513 (+ 583us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:30:29.483684 (+ 171us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:30:29.483778 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:29.483939 (+ 161us) server_negotiation.cc:299] Negotiation successful
0811 06:30:29.484023 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 06:30:29.485620 10126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:29.476581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.193:42829 (local address 127.28.239.193:35439)
0811 06:30:29.477357 (+ 776us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:29.477383 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:29.477561 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:29.477971 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:29.477977 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:29.477989 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 06:30:29.478398 (+ 409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:29.478410 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:29.479957 (+ 1547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:29.479963 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:29.481932 (+ 1969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:29.481949 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:29.483032 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:29.483039 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:29.483239 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:29.483314 (+ 75us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 06:30:29.483820 (+ 506us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 06:30:29.483830 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 06:30:29.484604 (+ 774us) client_negotiation.cc:240] Negotiation successful
0811 06:30:29.485143 (+ 539us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":614,"thread_start_us":435,"threads_started":1}
I20260811 06:30:29.501235 9939 raft_consensus.cc:1261] T 5a4e01e0edc747c7bd2fad848ed47824 P 34c9ac0ce67d4df5863681360f27c4ac [term 1 FOLLOWER]: Refusing update from remote peer 57b2448bb01c461a964ee8ceca7f2315: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:30:29.501987 10093 consensus_queue.cc:1035] T 5a4e01e0edc747c7bd2fad848ed47824 P 57b2448bb01c461a964ee8ceca7f2315 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34c9ac0ce67d4df5863681360f27c4ac" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 37827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:30:29.502952 9746 raft_consensus.cc:1261] T 5a4e01e0edc747c7bd2fad848ed47824 P 2e276abb6ae14f2aab047be5cb9cb373 [term 1 FOLLOWER]: Refusing update from remote peer 57b2448bb01c461a964ee8ceca7f2315: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:30:29.503643 10093 consensus_queue.cc:1035] T 5a4e01e0edc747c7bd2fad848ed47824 P 57b2448bb01c461a964ee8ceca7f2315 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e276abb6ae14f2aab047be5cb9cb373" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:30:29.513053 10068 mvcc.cc:205] Tried to move back new op lower bound from 7317216581627256832 to 7317216580681920512. Current Snapshot: MvccSnapshot[applied={T|T < 7317216581627256832}]
I20260811 06:30:29.590837 9265 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33940:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:30:29.591281 9265 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33940:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:30:29.595840 9265 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2678652809584c6cb6c707b112de9a62: Sending DeleteTablet for 3 replicas of tablet 5a4e01e0edc747c7bd2fad848ed47824
I20260811 06:30:29.597688 9490 tablet_service.cc:1514] Processing DeleteTablet for tablet 5a4e01e0edc747c7bd2fad848ed47824 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:30:29 UTC) from {username='oryx', principal='oryx/127.28.239.254@KRBTEST.COM'} at 127.0.0.1:51296
I20260811 06:30:29.598182 10085 tablet_replica.cc:330] T 5a4e01e0edc747c7bd2fad848ed47824 P 57b2448bb01c461a964ee8ceca7f2315: stopping tablet replica
I20260811 06:30:29.598445 10085 raft_consensus.cc:2229] T 5a4e01e0edc747c7bd2fad848ed47824 P 57b2448bb01c461a964ee8ceca7f2315 [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:30:29.599074 10085 raft_consensus.cc:2258] T 5a4e01e0edc747c7bd2fad848ed47824 P 57b2448bb01c461a964ee8ceca7f2315 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:30:29.599469 9705 tablet_service.cc:1514] Processing DeleteTablet for tablet 5a4e01e0edc747c7bd2fad848ed47824 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:30:29 UTC) from {username='oryx', principal='oryx/127.28.239.254@KRBTEST.COM'} at 127.0.0.1:42208
I20260811 06:30:29.599748 9910 tablet_service.cc:1514] Processing DeleteTablet for tablet 5a4e01e0edc747c7bd2fad848ed47824 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:30:29 UTC) from {username='oryx', principal='oryx/127.28.239.254@KRBTEST.COM'} at 127.0.0.1:42686
I20260811 06:30:29.600010 10086 tablet_replica.cc:330] T 5a4e01e0edc747c7bd2fad848ed47824 P 2e276abb6ae14f2aab047be5cb9cb373: stopping tablet replica
I20260811 06:30:29.600116 10084 tablet_replica.cc:330] T 5a4e01e0edc747c7bd2fad848ed47824 P 34c9ac0ce67d4df5863681360f27c4ac: stopping tablet replica
I20260811 06:30:29.600231 10086 raft_consensus.cc:2229] T 5a4e01e0edc747c7bd2fad848ed47824 P 2e276abb6ae14f2aab047be5cb9cb373 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:30:29.600320 10084 raft_consensus.cc:2229] T 5a4e01e0edc747c7bd2fad848ed47824 P 34c9ac0ce67d4df5863681360f27c4ac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:30:29.600437 10086 raft_consensus.cc:2258] T 5a4e01e0edc747c7bd2fad848ed47824 P 2e276abb6ae14f2aab047be5cb9cb373 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:30:29.600566 10084 raft_consensus.cc:2258] T 5a4e01e0edc747c7bd2fad848ed47824 P 34c9ac0ce67d4df5863681360f27c4ac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:30:29.602178 10086 ts_tablet_manager.cc:1905] T 5a4e01e0edc747c7bd2fad848ed47824 P 2e276abb6ae14f2aab047be5cb9cb373: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:30:29.602339 10084 ts_tablet_manager.cc:1905] T 5a4e01e0edc747c7bd2fad848ed47824 P 34c9ac0ce67d4df5863681360f27c4ac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:30:29.602939 10085 ts_tablet_manager.cc:1905] T 5a4e01e0edc747c7bd2fad848ed47824 P 57b2448bb01c461a964ee8ceca7f2315: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:30:29.609416 10086 ts_tablet_manager.cc:1918] T 5a4e01e0edc747c7bd2fad848ed47824 P 2e276abb6ae14f2aab047be5cb9cb373: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 06:30:29.609603 10086 log.cc:1196] T 5a4e01e0edc747c7bd2fad848ed47824 P 2e276abb6ae14f2aab047be5cb9cb373: Deleting WAL directory at /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/minicluster-data/ts-1/wal/wals/5a4e01e0edc747c7bd2fad848ed47824
I20260811 06:30:29.610075 10086 ts_tablet_manager.cc:1939] T 5a4e01e0edc747c7bd2fad848ed47824 P 2e276abb6ae14f2aab047be5cb9cb373: Deleting consensus metadata
I20260811 06:30:29.610442 10084 ts_tablet_manager.cc:1918] T 5a4e01e0edc747c7bd2fad848ed47824 P 34c9ac0ce67d4df5863681360f27c4ac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 06:30:29.610638 10084 log.cc:1196] T 5a4e01e0edc747c7bd2fad848ed47824 P 34c9ac0ce67d4df5863681360f27c4ac: Deleting WAL directory at /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/minicluster-data/ts-2/wal/wals/5a4e01e0edc747c7bd2fad848ed47824
I20260811 06:30:29.611076 10084 ts_tablet_manager.cc:1939] T 5a4e01e0edc747c7bd2fad848ed47824 P 34c9ac0ce67d4df5863681360f27c4ac: Deleting consensus metadata
I20260811 06:30:29.611524 9244 catalog_manager.cc:4841] TS 2e276abb6ae14f2aab047be5cb9cb373 (127.28.239.194:37681): tablet 5a4e01e0edc747c7bd2fad848ed47824 (table test-table [id=0829373076a74d13bb2f80ffca798d12]) successfully deleted
I20260811 06:30:29.612943 9242 catalog_manager.cc:4841] TS 34c9ac0ce67d4df5863681360f27c4ac (127.28.239.195:37827): tablet 5a4e01e0edc747c7bd2fad848ed47824 (table test-table [id=0829373076a74d13bb2f80ffca798d12]) successfully deleted
I20260811 06:30:29.613162 10085 ts_tablet_manager.cc:1918] T 5a4e01e0edc747c7bd2fad848ed47824 P 57b2448bb01c461a964ee8ceca7f2315: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 06:30:29.613320 10085 log.cc:1196] T 5a4e01e0edc747c7bd2fad848ed47824 P 57b2448bb01c461a964ee8ceca7f2315: Deleting WAL directory at /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/minicluster-data/ts-0/wal/wals/5a4e01e0edc747c7bd2fad848ed47824
I20260811 06:30:29.613828 10085 ts_tablet_manager.cc:1939] T 5a4e01e0edc747c7bd2fad848ed47824 P 57b2448bb01c461a964ee8ceca7f2315: Deleting consensus metadata
I20260811 06:30:29.615638 9244 catalog_manager.cc:4841] TS 57b2448bb01c461a964ee8ceca7f2315 (127.28.239.193:42829): tablet 5a4e01e0edc747c7bd2fad848ed47824 (table test-table [id=0829373076a74d13bb2f80ffca798d12]) successfully deleted
Aug 11 06:30:29 dist-test-slave-gkw7 krb5kdc[9201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429829, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 06:30:29 dist-test-slave-gkw7 krb5kdc[9201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:30:29 dist-test-slave-gkw7 krb5kdc[9201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
W20260811 06:30:29.712474 9852 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:30:29.697845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33950 (local address 127.28.239.254:46215)
0811 06:30:29.698077 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:29.698084 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:29.698733 (+ 649us) server_negotiation.cc:407] Connection header received
0811 06:30:29.698961 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:29.698967 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:29.699059 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:29.699186 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:29.701162 (+ 1976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:29.702477 (+ 1315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:29.704573 (+ 2096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:29.705260 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:29.705391 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:29.712197 (+ 6806us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33950: BlockingRecv error: recv got EOF from 127.0.0.1:33950 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Aug 11 06:30:29 dist-test-slave-gkw7 krb5kdc[9201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429829, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM
I20260811 06:30:29.794839 9852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:29.759135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33954 (local address 127.28.239.254:46215)
0811 06:30:29.759366 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:29.759373 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:29.773380 (+ 14007us) server_negotiation.cc:407] Connection header received
0811 06:30:29.773587 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:29.773594 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:29.773723 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:29.773857 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:29.776061 (+ 2204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:29.777314 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:29.779505 (+ 2191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:29.780258 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:29.780377 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:29.786453 (+ 6076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:29.786462 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:29.786467 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:30:29.786522 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:29.790663 (+ 4141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:29.791739 (+ 1076us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:29.791745 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:29.791750 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:29.791934 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:29.792626 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:29.792632 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:29.792636 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:29.793062 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:29.793365 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:29.794091 (+ 726us) server_negotiation.cc:299] Negotiation successful
0811 06:30:29.794395 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 06:30:29.800419 9265 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33954:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:30:29.801162 9265 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:30:29.810370 9910 tablet_service.cc:1467] Processing CreateTablet for tablet 92c249d18dc04ef0a3cff30158affd53 (DEFAULT_TABLE table=test-table [id=424c53fa5ead4c32855c63bc86fe8f83]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:30:29.810986 9910 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92c249d18dc04ef0a3cff30158affd53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:30:29.811399 9705 tablet_service.cc:1467] Processing CreateTablet for tablet 92c249d18dc04ef0a3cff30158affd53 (DEFAULT_TABLE table=test-table [id=424c53fa5ead4c32855c63bc86fe8f83]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:30:29.811887 9705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92c249d18dc04ef0a3cff30158affd53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:30:29.818156 9490 tablet_service.cc:1467] Processing CreateTablet for tablet 92c249d18dc04ef0a3cff30158affd53 (DEFAULT_TABLE table=test-table [id=424c53fa5ead4c32855c63bc86fe8f83]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:30:29.818570 10049 tablet_bootstrap.cc:492] T 92c249d18dc04ef0a3cff30158affd53 P 2e276abb6ae14f2aab047be5cb9cb373: Bootstrap starting.
I20260811 06:30:29.818601 9490 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92c249d18dc04ef0a3cff30158affd53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:30:29.819567 10051 tablet_bootstrap.cc:492] T 92c249d18dc04ef0a3cff30158affd53 P 34c9ac0ce67d4df5863681360f27c4ac: Bootstrap starting.
I20260811 06:30:29.820245 10049 tablet_bootstrap.cc:654] T 92c249d18dc04ef0a3cff30158affd53 P 2e276abb6ae14f2aab047be5cb9cb373: Neither blocks nor log segments found. Creating new log.
I20260811 06:30:29.821498 10051 tablet_bootstrap.cc:654] T 92c249d18dc04ef0a3cff30158affd53 P 34c9ac0ce67d4df5863681360f27c4ac: Neither blocks nor log segments found. Creating new log.
I20260811 06:30:29.826562 10050 tablet_bootstrap.cc:492] T 92c249d18dc04ef0a3cff30158affd53 P 57b2448bb01c461a964ee8ceca7f2315: Bootstrap starting.
I20260811 06:30:29.828780 10050 tablet_bootstrap.cc:654] T 92c249d18dc04ef0a3cff30158affd53 P 57b2448bb01c461a964ee8ceca7f2315: Neither blocks nor log segments found. Creating new log.
I20260811 06:30:29.829635 10051 tablet_bootstrap.cc:492] T 92c249d18dc04ef0a3cff30158affd53 P 34c9ac0ce67d4df5863681360f27c4ac: No bootstrap required, opened a new log
I20260811 06:30:29.829834 10051 ts_tablet_manager.cc:1397] T 92c249d18dc04ef0a3cff30158affd53 P 34c9ac0ce67d4df5863681360f27c4ac: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.001s
I20260811 06:30:29.830873 10051 raft_consensus.cc:348] T 92c249d18dc04ef0a3cff30158affd53 P 34c9ac0ce67d4df5863681360f27c4ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b2448bb01c461a964ee8ceca7f2315" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 42829 } } peers { permanent_uuid: "2e276abb6ae14f2aab047be5cb9cb373" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37681 } } peers { permanent_uuid: "34c9ac0ce67d4df5863681360f27c4ac" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 37827 } }
I20260811 06:30:29.831250 10051 raft_consensus.cc:374] T 92c249d18dc04ef0a3cff30158affd53 P 34c9ac0ce67d4df5863681360f27c4ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:30:29.831375 10051 raft_consensus.cc:729] T 92c249d18dc04ef0a3cff30158affd53 P 34c9ac0ce67d4df5863681360f27c4ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34c9ac0ce67d4df5863681360f27c4ac, State: Initialized, Role: FOLLOWER
I20260811 06:30:29.831661 10051 consensus_queue.cc:260] T 92c249d18dc04ef0a3cff30158affd53 P 34c9ac0ce67d4df5863681360f27c4ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b2448bb01c461a964ee8ceca7f2315" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 42829 } } peers { permanent_uuid: "2e276abb6ae14f2aab047be5cb9cb373" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37681 } } peers { permanent_uuid: "34c9ac0ce67d4df5863681360f27c4ac" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 37827 } }
I20260811 06:30:29.836285 10051 ts_tablet_manager.cc:1428] T 92c249d18dc04ef0a3cff30158affd53 P 34c9ac0ce67d4df5863681360f27c4ac: Time spent starting tablet: real 0.006s user 0.000s sys 0.002s
I20260811 06:30:29.840571 10049 tablet_bootstrap.cc:492] T 92c249d18dc04ef0a3cff30158affd53 P 2e276abb6ae14f2aab047be5cb9cb373: No bootstrap required, opened a new log
I20260811 06:30:29.840940 10049 ts_tablet_manager.cc:1397] T 92c249d18dc04ef0a3cff30158affd53 P 2e276abb6ae14f2aab047be5cb9cb373: Time spent bootstrapping tablet: real 0.022s user 0.004s sys 0.004s
I20260811 06:30:29.842097 10049 raft_consensus.cc:348] T 92c249d18dc04ef0a3cff30158affd53 P 2e276abb6ae14f2aab047be5cb9cb373 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b2448bb01c461a964ee8ceca7f2315" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 42829 } } peers { permanent_uuid: "2e276abb6ae14f2aab047be5cb9cb373" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37681 } } peers { permanent_uuid: "34c9ac0ce67d4df5863681360f27c4ac" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 37827 } }
I20260811 06:30:29.842396 10049 raft_consensus.cc:374] T 92c249d18dc04ef0a3cff30158affd53 P 2e276abb6ae14f2aab047be5cb9cb373 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:30:29.842501 10049 raft_consensus.cc:729] T 92c249d18dc04ef0a3cff30158affd53 P 2e276abb6ae14f2aab047be5cb9cb373 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e276abb6ae14f2aab047be5cb9cb373, State: Initialized, Role: FOLLOWER
I20260811 06:30:29.842751 10049 consensus_queue.cc:260] T 92c249d18dc04ef0a3cff30158affd53 P 2e276abb6ae14f2aab047be5cb9cb373 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b2448bb01c461a964ee8ceca7f2315" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 42829 } } peers { permanent_uuid: "2e276abb6ae14f2aab047be5cb9cb373" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37681 } } peers { permanent_uuid: "34c9ac0ce67d4df5863681360f27c4ac" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 37827 } }
I20260811 06:30:29.843354 10049 ts_tablet_manager.cc:1428] T 92c249d18dc04ef0a3cff30158affd53 P 2e276abb6ae14f2aab047be5cb9cb373: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260811 06:30:29.847393 10050 tablet_bootstrap.cc:492] T 92c249d18dc04ef0a3cff30158affd53 P 57b2448bb01c461a964ee8ceca7f2315: No bootstrap required, opened a new log
I20260811 06:30:29.847615 10050 ts_tablet_manager.cc:1397] T 92c249d18dc04ef0a3cff30158affd53 P 57b2448bb01c461a964ee8ceca7f2315: Time spent bootstrapping tablet: real 0.021s user 0.005s sys 0.009s
I20260811 06:30:29.848774 10050 raft_consensus.cc:348] T 92c249d18dc04ef0a3cff30158affd53 P 57b2448bb01c461a964ee8ceca7f2315 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b2448bb01c461a964ee8ceca7f2315" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 42829 } } peers { permanent_uuid: "2e276abb6ae14f2aab047be5cb9cb373" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37681 } } peers { permanent_uuid: "34c9ac0ce67d4df5863681360f27c4ac" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 37827 } }
I20260811 06:30:29.849069 10050 raft_consensus.cc:374] T 92c249d18dc04ef0a3cff30158affd53 P 57b2448bb01c461a964ee8ceca7f2315 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:30:29.849193 10050 raft_consensus.cc:729] T 92c249d18dc04ef0a3cff30158affd53 P 57b2448bb01c461a964ee8ceca7f2315 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57b2448bb01c461a964ee8ceca7f2315, State: Initialized, Role: FOLLOWER
I20260811 06:30:29.849493 10050 consensus_queue.cc:260] T 92c249d18dc04ef0a3cff30158affd53 P 57b2448bb01c461a964ee8ceca7f2315 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b2448bb01c461a964ee8ceca7f2315" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 42829 } } peers { permanent_uuid: "2e276abb6ae14f2aab047be5cb9cb373" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37681 } } peers { permanent_uuid: "34c9ac0ce67d4df5863681360f27c4ac" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 37827 } }
I20260811 06:30:29.850768 10050 ts_tablet_manager.cc:1428] T 92c249d18dc04ef0a3cff30158affd53 P 57b2448bb01c461a964ee8ceca7f2315: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 06:30:29.854715 10060 raft_consensus.cc:482] T 92c249d18dc04ef0a3cff30158affd53 P 57b2448bb01c461a964ee8ceca7f2315 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:30:29.854912 10060 raft_consensus.cc:504] T 92c249d18dc04ef0a3cff30158affd53 P 57b2448bb01c461a964ee8ceca7f2315 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b2448bb01c461a964ee8ceca7f2315" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 42829 } } peers { permanent_uuid: "2e276abb6ae14f2aab047be5cb9cb373" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37681 } } peers { permanent_uuid: "34c9ac0ce67d4df5863681360f27c4ac" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 37827 } }
I20260811 06:30:29.855736 10060 leader_election.cc:288] T 92c249d18dc04ef0a3cff30158affd53 P 57b2448bb01c461a964ee8ceca7f2315 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2e276abb6ae14f2aab047be5cb9cb373 (127.28.239.194:37681), 34c9ac0ce67d4df5863681360f27c4ac (127.28.239.195:37827)
I20260811 06:30:29.856230 9746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92c249d18dc04ef0a3cff30158affd53" candidate_uuid: "57b2448bb01c461a964ee8ceca7f2315" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e276abb6ae14f2aab047be5cb9cb373" is_pre_election: true
I20260811 06:30:29.856508 9746 raft_consensus.cc:2454] T 92c249d18dc04ef0a3cff30158affd53 P 2e276abb6ae14f2aab047be5cb9cb373 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57b2448bb01c461a964ee8ceca7f2315 in term 0.
I20260811 06:30:29.856976 9939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92c249d18dc04ef0a3cff30158affd53" candidate_uuid: "57b2448bb01c461a964ee8ceca7f2315" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34c9ac0ce67d4df5863681360f27c4ac" is_pre_election: true
I20260811 06:30:29.857039 9413 leader_election.cc:302] T 92c249d18dc04ef0a3cff30158affd53 P 57b2448bb01c461a964ee8ceca7f2315 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e276abb6ae14f2aab047be5cb9cb373, 57b2448bb01c461a964ee8ceca7f2315; no voters:
I20260811 06:30:29.857277 9939 raft_consensus.cc:2454] T 92c249d18dc04ef0a3cff30158affd53 P 34c9ac0ce67d4df5863681360f27c4ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57b2448bb01c461a964ee8ceca7f2315 in term 0.
I20260811 06:30:29.857419 10060 raft_consensus.cc:2781] T 92c249d18dc04ef0a3cff30158affd53 P 57b2448bb01c461a964ee8ceca7f2315 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:30:29.857580 10060 raft_consensus.cc:482] T 92c249d18dc04ef0a3cff30158affd53 P 57b2448bb01c461a964ee8ceca7f2315 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:30:29.857672 10060 raft_consensus.cc:3037] T 92c249d18dc04ef0a3cff30158affd53 P 57b2448bb01c461a964ee8ceca7f2315 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:30:29.860875 10060 raft_consensus.cc:504] T 92c249d18dc04ef0a3cff30158affd53 P 57b2448bb01c461a964ee8ceca7f2315 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b2448bb01c461a964ee8ceca7f2315" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 42829 } } peers { permanent_uuid: "2e276abb6ae14f2aab047be5cb9cb373" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37681 } } peers { permanent_uuid: "34c9ac0ce67d4df5863681360f27c4ac" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 37827 } }
I20260811 06:30:29.861696 10060 leader_election.cc:288] T 92c249d18dc04ef0a3cff30158affd53 P 57b2448bb01c461a964ee8ceca7f2315 [CANDIDATE]: Term 1 election: Requested vote from peers 2e276abb6ae14f2aab047be5cb9cb373 (127.28.239.194:37681), 34c9ac0ce67d4df5863681360f27c4ac (127.28.239.195:37827)
I20260811 06:30:29.862063 9939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92c249d18dc04ef0a3cff30158affd53" candidate_uuid: "57b2448bb01c461a964ee8ceca7f2315" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34c9ac0ce67d4df5863681360f27c4ac"
I20260811 06:30:29.862272 9939 raft_consensus.cc:3037] T 92c249d18dc04ef0a3cff30158affd53 P 34c9ac0ce67d4df5863681360f27c4ac [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:30:29.862680 9746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92c249d18dc04ef0a3cff30158affd53" candidate_uuid: "57b2448bb01c461a964ee8ceca7f2315" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e276abb6ae14f2aab047be5cb9cb373"
I20260811 06:30:29.862914 9746 raft_consensus.cc:3037] T 92c249d18dc04ef0a3cff30158affd53 P 2e276abb6ae14f2aab047be5cb9cb373 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:30:29.865346 9939 raft_consensus.cc:2454] T 92c249d18dc04ef0a3cff30158affd53 P 34c9ac0ce67d4df5863681360f27c4ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57b2448bb01c461a964ee8ceca7f2315 in term 1.
I20260811 06:30:29.866178 9411 leader_election.cc:302] T 92c249d18dc04ef0a3cff30158affd53 P 57b2448bb01c461a964ee8ceca7f2315 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34c9ac0ce67d4df5863681360f27c4ac, 57b2448bb01c461a964ee8ceca7f2315; no voters:
I20260811 06:30:29.866775 10060 raft_consensus.cc:2781] T 92c249d18dc04ef0a3cff30158affd53 P 57b2448bb01c461a964ee8ceca7f2315 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:30:29.867020 10060 raft_consensus.cc:686] T 92c249d18dc04ef0a3cff30158affd53 P 57b2448bb01c461a964ee8ceca7f2315 [term 1 LEADER]: Becoming Leader. State: Replica: 57b2448bb01c461a964ee8ceca7f2315, State: Running, Role: LEADER
I20260811 06:30:29.867331 10060 consensus_queue.cc:237] T 92c249d18dc04ef0a3cff30158affd53 P 57b2448bb01c461a964ee8ceca7f2315 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b2448bb01c461a964ee8ceca7f2315" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 42829 } } peers { permanent_uuid: "2e276abb6ae14f2aab047be5cb9cb373" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37681 } } peers { permanent_uuid: "34c9ac0ce67d4df5863681360f27c4ac" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 37827 } }
I20260811 06:30:29.868180 9746 raft_consensus.cc:2454] T 92c249d18dc04ef0a3cff30158affd53 P 2e276abb6ae14f2aab047be5cb9cb373 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57b2448bb01c461a964ee8ceca7f2315 in term 1.
I20260811 06:30:29.870864 9267 catalog_manager.cc:5495] T 92c249d18dc04ef0a3cff30158affd53 P 57b2448bb01c461a964ee8ceca7f2315 reported cstate change: term changed from 0 to 1, leader changed from <none> to 57b2448bb01c461a964ee8ceca7f2315 (127.28.239.193). New cstate: current_term: 1 leader_uuid: "57b2448bb01c461a964ee8ceca7f2315" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b2448bb01c461a964ee8ceca7f2315" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 42829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2e276abb6ae14f2aab047be5cb9cb373" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34c9ac0ce67d4df5863681360f27c4ac" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 37827 } health_report { overall_health: UNKNOWN } } }
I20260811 06:30:29.904182 10045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:29.896085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51332 (local address 127.28.239.193:42829)
0811 06:30:29.896293 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:29.896301 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:29.896891 (+ 590us) server_negotiation.cc:407] Connection header received
0811 06:30:29.897129 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:29.897135 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:29.897263 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:29.897370 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:30:29.898047 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:29.899327 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:29.901566 (+ 2239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:29.902333 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:29.902455 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:29.903069 (+ 614us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:30:29.903231 (+ 162us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:30:29.903314 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:29.903770 (+ 456us) server_negotiation.cc:299] Negotiation successful
0811 06:30:29.903851 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 06:30:29.908080 9746 raft_consensus.cc:1261] T 92c249d18dc04ef0a3cff30158affd53 P 2e276abb6ae14f2aab047be5cb9cb373 [term 1 FOLLOWER]: Refusing update from remote peer 57b2448bb01c461a964ee8ceca7f2315: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:30:29.909351 9939 raft_consensus.cc:1261] T 92c249d18dc04ef0a3cff30158affd53 P 34c9ac0ce67d4df5863681360f27c4ac [term 1 FOLLOWER]: Refusing update from remote peer 57b2448bb01c461a964ee8ceca7f2315: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:30:29.910014 10093 consensus_queue.cc:1035] T 92c249d18dc04ef0a3cff30158affd53 P 57b2448bb01c461a964ee8ceca7f2315 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e276abb6ae14f2aab047be5cb9cb373" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:30:29.911484 10093 consensus_queue.cc:1035] T 92c249d18dc04ef0a3cff30158affd53 P 57b2448bb01c461a964ee8ceca7f2315 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34c9ac0ce67d4df5863681360f27c4ac" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 37827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:30:29.931994 10069 mvcc.cc:205] Tried to move back new op lower bound from 7317216583294570496 to 7317216583139131392. Current Snapshot: MvccSnapshot[applied={T|T < 7317216583294570496}]
I20260811 06:30:29.942727 9265 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33954:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:30:29.942939 9265 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33954:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:30:29.946369 9265 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2678652809584c6cb6c707b112de9a62: Sending DeleteTablet for 3 replicas of tablet 92c249d18dc04ef0a3cff30158affd53
I20260811 06:30:29.947299 9490 tablet_service.cc:1514] Processing DeleteTablet for tablet 92c249d18dc04ef0a3cff30158affd53 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:30:29 UTC) from {username='oryx', principal='oryx/127.28.239.254@KRBTEST.COM'} at 127.0.0.1:51296
I20260811 06:30:29.947431 9705 tablet_service.cc:1514] Processing DeleteTablet for tablet 92c249d18dc04ef0a3cff30158affd53 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:30:29 UTC) from {username='oryx', principal='oryx/127.28.239.254@KRBTEST.COM'} at 127.0.0.1:42208
I20260811 06:30:29.947672 10085 tablet_replica.cc:330] T 92c249d18dc04ef0a3cff30158affd53 P 57b2448bb01c461a964ee8ceca7f2315: stopping tablet replica
I20260811 06:30:29.947794 10086 tablet_replica.cc:330] T 92c249d18dc04ef0a3cff30158affd53 P 2e276abb6ae14f2aab047be5cb9cb373: stopping tablet replica
I20260811 06:30:29.947803 9910 tablet_service.cc:1514] Processing DeleteTablet for tablet 92c249d18dc04ef0a3cff30158affd53 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:30:29 UTC) from {username='oryx', principal='oryx/127.28.239.254@KRBTEST.COM'} at 127.0.0.1:42686
I20260811 06:30:29.947888 10085 raft_consensus.cc:2229] T 92c249d18dc04ef0a3cff30158affd53 P 57b2448bb01c461a964ee8ceca7f2315 [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:30:29.948001 10086 raft_consensus.cc:2229] T 92c249d18dc04ef0a3cff30158affd53 P 2e276abb6ae14f2aab047be5cb9cb373 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:30:29.948109 10084 tablet_replica.cc:330] T 92c249d18dc04ef0a3cff30158affd53 P 34c9ac0ce67d4df5863681360f27c4ac: stopping tablet replica
I20260811 06:30:29.948272 10085 raft_consensus.cc:2258] T 92c249d18dc04ef0a3cff30158affd53 P 57b2448bb01c461a964ee8ceca7f2315 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:30:29.948278 10086 raft_consensus.cc:2258] T 92c249d18dc04ef0a3cff30158affd53 P 2e276abb6ae14f2aab047be5cb9cb373 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:30:29.948319 10084 raft_consensus.cc:2229] T 92c249d18dc04ef0a3cff30158affd53 P 34c9ac0ce67d4df5863681360f27c4ac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:30:29.948567 10084 raft_consensus.cc:2258] T 92c249d18dc04ef0a3cff30158affd53 P 34c9ac0ce67d4df5863681360f27c4ac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:30:29.949476 10085 ts_tablet_manager.cc:1905] T 92c249d18dc04ef0a3cff30158affd53 P 57b2448bb01c461a964ee8ceca7f2315: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:30:29.949822 10084 ts_tablet_manager.cc:1905] T 92c249d18dc04ef0a3cff30158affd53 P 34c9ac0ce67d4df5863681360f27c4ac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:30:29.953492 10086 ts_tablet_manager.cc:1905] T 92c249d18dc04ef0a3cff30158affd53 P 2e276abb6ae14f2aab047be5cb9cb373: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:30:29.954423 9265 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33954:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 06:30:29.956067 10084 ts_tablet_manager.cc:1918] T 92c249d18dc04ef0a3cff30158affd53 P 34c9ac0ce67d4df5863681360f27c4ac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:30:29.956228 10084 log.cc:1196] T 92c249d18dc04ef0a3cff30158affd53 P 34c9ac0ce67d4df5863681360f27c4ac: Deleting WAL directory at /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/minicluster-data/ts-2/wal/wals/92c249d18dc04ef0a3cff30158affd53
I20260811 06:30:29.956753 10084 ts_tablet_manager.cc:1939] T 92c249d18dc04ef0a3cff30158affd53 P 34c9ac0ce67d4df5863681360f27c4ac: Deleting consensus metadata
W20260811 06:30:29.956979 9265 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:30:29.957981 9242 catalog_manager.cc:4841] TS 34c9ac0ce67d4df5863681360f27c4ac (127.28.239.195:37827): tablet 92c249d18dc04ef0a3cff30158affd53 (table test-table [id=424c53fa5ead4c32855c63bc86fe8f83]) successfully deleted
I20260811 06:30:29.960539 10086 ts_tablet_manager.cc:1918] T 92c249d18dc04ef0a3cff30158affd53 P 2e276abb6ae14f2aab047be5cb9cb373: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:30:29.960708 10086 log.cc:1196] T 92c249d18dc04ef0a3cff30158affd53 P 2e276abb6ae14f2aab047be5cb9cb373: Deleting WAL directory at /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/minicluster-data/ts-1/wal/wals/92c249d18dc04ef0a3cff30158affd53
I20260811 06:30:29.961202 10086 ts_tablet_manager.cc:1939] T 92c249d18dc04ef0a3cff30158affd53 P 2e276abb6ae14f2aab047be5cb9cb373: Deleting consensus metadata
I20260811 06:30:29.961808 10085 ts_tablet_manager.cc:1918] T 92c249d18dc04ef0a3cff30158affd53 P 57b2448bb01c461a964ee8ceca7f2315: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:30:29.962028 10085 log.cc:1196] T 92c249d18dc04ef0a3cff30158affd53 P 57b2448bb01c461a964ee8ceca7f2315: Deleting WAL directory at /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/minicluster-data/ts-0/wal/wals/92c249d18dc04ef0a3cff30158affd53
I20260811 06:30:29.962514 9244 catalog_manager.cc:4841] TS 2e276abb6ae14f2aab047be5cb9cb373 (127.28.239.194:37681): tablet 92c249d18dc04ef0a3cff30158affd53 (table test-table [id=424c53fa5ead4c32855c63bc86fe8f83]) successfully deleted
I20260811 06:30:29.962591 10085 ts_tablet_manager.cc:1939] T 92c249d18dc04ef0a3cff30158affd53 P 57b2448bb01c461a964ee8ceca7f2315: Deleting consensus metadata
I20260811 06:30:29.963894 9244 catalog_manager.cc:4841] TS 57b2448bb01c461a964ee8ceca7f2315 (127.28.239.193:42829): tablet 92c249d18dc04ef0a3cff30158affd53 (table test-table [id=424c53fa5ead4c32855c63bc86fe8f83]) successfully deleted
I20260811 06:30:29.965649 9490 tablet_service.cc:1467] Processing CreateTablet for tablet 30e10a0693144530ab60b8070ff243cc (DEFAULT_TABLE table=test-table [id=c2cc565794914868a96b9e02d1ab1229]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:30:29.966133 9490 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30e10a0693144530ab60b8070ff243cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:30:29.966125 9705 tablet_service.cc:1467] Processing CreateTablet for tablet 30e10a0693144530ab60b8070ff243cc (DEFAULT_TABLE table=test-table [id=c2cc565794914868a96b9e02d1ab1229]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:30:29.966154 9910 tablet_service.cc:1467] Processing CreateTablet for tablet 30e10a0693144530ab60b8070ff243cc (DEFAULT_TABLE table=test-table [id=c2cc565794914868a96b9e02d1ab1229]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:30:29.966531 9705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30e10a0693144530ab60b8070ff243cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:30:29.966655 9910 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30e10a0693144530ab60b8070ff243cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:30:29.973158 10049 tablet_bootstrap.cc:492] T 30e10a0693144530ab60b8070ff243cc P 2e276abb6ae14f2aab047be5cb9cb373: Bootstrap starting.
I20260811 06:30:29.973228 10050 tablet_bootstrap.cc:492] T 30e10a0693144530ab60b8070ff243cc P 57b2448bb01c461a964ee8ceca7f2315: Bootstrap starting.
I20260811 06:30:29.973767 10051 tablet_bootstrap.cc:492] T 30e10a0693144530ab60b8070ff243cc P 34c9ac0ce67d4df5863681360f27c4ac: Bootstrap starting.
I20260811 06:30:29.974965 10050 tablet_bootstrap.cc:654] T 30e10a0693144530ab60b8070ff243cc P 57b2448bb01c461a964ee8ceca7f2315: Neither blocks nor log segments found. Creating new log.
I20260811 06:30:29.975086 10049 tablet_bootstrap.cc:654] T 30e10a0693144530ab60b8070ff243cc P 2e276abb6ae14f2aab047be5cb9cb373: Neither blocks nor log segments found. Creating new log.
I20260811 06:30:29.975560 10051 tablet_bootstrap.cc:654] T 30e10a0693144530ab60b8070ff243cc P 34c9ac0ce67d4df5863681360f27c4ac: Neither blocks nor log segments found. Creating new log.
I20260811 06:30:29.978019 10049 tablet_bootstrap.cc:492] T 30e10a0693144530ab60b8070ff243cc P 2e276abb6ae14f2aab047be5cb9cb373: No bootstrap required, opened a new log
I20260811 06:30:29.978259 10049 ts_tablet_manager.cc:1397] T 30e10a0693144530ab60b8070ff243cc P 2e276abb6ae14f2aab047be5cb9cb373: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.001s
I20260811 06:30:29.978754 10050 tablet_bootstrap.cc:492] T 30e10a0693144530ab60b8070ff243cc P 57b2448bb01c461a964ee8ceca7f2315: No bootstrap required, opened a new log
I20260811 06:30:29.978988 10050 ts_tablet_manager.cc:1397] T 30e10a0693144530ab60b8070ff243cc P 57b2448bb01c461a964ee8ceca7f2315: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 06:30:29.979517 10049 raft_consensus.cc:348] T 30e10a0693144530ab60b8070ff243cc P 2e276abb6ae14f2aab047be5cb9cb373 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b2448bb01c461a964ee8ceca7f2315" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 42829 } } peers { permanent_uuid: "2e276abb6ae14f2aab047be5cb9cb373" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37681 } } peers { permanent_uuid: "34c9ac0ce67d4df5863681360f27c4ac" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 37827 } }
I20260811 06:30:29.980178 10049 raft_consensus.cc:374] T 30e10a0693144530ab60b8070ff243cc P 2e276abb6ae14f2aab047be5cb9cb373 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:30:29.980329 10049 raft_consensus.cc:729] T 30e10a0693144530ab60b8070ff243cc P 2e276abb6ae14f2aab047be5cb9cb373 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e276abb6ae14f2aab047be5cb9cb373, State: Initialized, Role: FOLLOWER
I20260811 06:30:29.980703 10049 consensus_queue.cc:260] T 30e10a0693144530ab60b8070ff243cc P 2e276abb6ae14f2aab047be5cb9cb373 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b2448bb01c461a964ee8ceca7f2315" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 42829 } } peers { permanent_uuid: "2e276abb6ae14f2aab047be5cb9cb373" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37681 } } peers { permanent_uuid: "34c9ac0ce67d4df5863681360f27c4ac" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 37827 } }
I20260811 06:30:29.980922 10050 raft_consensus.cc:348] T 30e10a0693144530ab60b8070ff243cc P 57b2448bb01c461a964ee8ceca7f2315 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b2448bb01c461a964ee8ceca7f2315" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 42829 } } peers { permanent_uuid: "2e276abb6ae14f2aab047be5cb9cb373" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37681 } } peers { permanent_uuid: "34c9ac0ce67d4df5863681360f27c4ac" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 37827 } }
I20260811 06:30:29.981312 10050 raft_consensus.cc:374] T 30e10a0693144530ab60b8070ff243cc P 57b2448bb01c461a964ee8ceca7f2315 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:30:29.981408 10050 raft_consensus.cc:729] T 30e10a0693144530ab60b8070ff243cc P 57b2448bb01c461a964ee8ceca7f2315 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57b2448bb01c461a964ee8ceca7f2315, State: Initialized, Role: FOLLOWER
I20260811 06:30:29.981772 10050 consensus_queue.cc:260] T 30e10a0693144530ab60b8070ff243cc P 57b2448bb01c461a964ee8ceca7f2315 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b2448bb01c461a964ee8ceca7f2315" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 42829 } } peers { permanent_uuid: "2e276abb6ae14f2aab047be5cb9cb373" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37681 } } peers { permanent_uuid: "34c9ac0ce67d4df5863681360f27c4ac" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 37827 } }
I20260811 06:30:29.983106 10049 ts_tablet_manager.cc:1428] T 30e10a0693144530ab60b8070ff243cc P 2e276abb6ae14f2aab047be5cb9cb373: Time spent starting tablet: real 0.005s user 0.000s sys 0.002s
I20260811 06:30:29.987082 10050 ts_tablet_manager.cc:1428] T 30e10a0693144530ab60b8070ff243cc P 57b2448bb01c461a964ee8ceca7f2315: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20260811 06:30:29.993258 10051 tablet_bootstrap.cc:492] T 30e10a0693144530ab60b8070ff243cc P 34c9ac0ce67d4df5863681360f27c4ac: No bootstrap required, opened a new log
I20260811 06:30:29.993468 10051 ts_tablet_manager.cc:1397] T 30e10a0693144530ab60b8070ff243cc P 34c9ac0ce67d4df5863681360f27c4ac: Time spent bootstrapping tablet: real 0.020s user 0.003s sys 0.015s
I20260811 06:30:29.994210 10051 raft_consensus.cc:348] T 30e10a0693144530ab60b8070ff243cc P 34c9ac0ce67d4df5863681360f27c4ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b2448bb01c461a964ee8ceca7f2315" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 42829 } } peers { permanent_uuid: "2e276abb6ae14f2aab047be5cb9cb373" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37681 } } peers { permanent_uuid: "34c9ac0ce67d4df5863681360f27c4ac" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 37827 } }
I20260811 06:30:29.994449 10051 raft_consensus.cc:374] T 30e10a0693144530ab60b8070ff243cc P 34c9ac0ce67d4df5863681360f27c4ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:30:29.994527 10051 raft_consensus.cc:729] T 30e10a0693144530ab60b8070ff243cc P 34c9ac0ce67d4df5863681360f27c4ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34c9ac0ce67d4df5863681360f27c4ac, State: Initialized, Role: FOLLOWER
I20260811 06:30:29.994831 10051 consensus_queue.cc:260] T 30e10a0693144530ab60b8070ff243cc P 34c9ac0ce67d4df5863681360f27c4ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b2448bb01c461a964ee8ceca7f2315" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 42829 } } peers { permanent_uuid: "2e276abb6ae14f2aab047be5cb9cb373" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37681 } } peers { permanent_uuid: "34c9ac0ce67d4df5863681360f27c4ac" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 37827 } }
I20260811 06:30:29.995368 10051 ts_tablet_manager.cc:1428] T 30e10a0693144530ab60b8070ff243cc P 34c9ac0ce67d4df5863681360f27c4ac: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260811 06:30:30.175834 10115 raft_consensus.cc:482] T 30e10a0693144530ab60b8070ff243cc P 57b2448bb01c461a964ee8ceca7f2315 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:30:30.176100 10115 raft_consensus.cc:504] T 30e10a0693144530ab60b8070ff243cc P 57b2448bb01c461a964ee8ceca7f2315 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b2448bb01c461a964ee8ceca7f2315" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 42829 } } peers { permanent_uuid: "2e276abb6ae14f2aab047be5cb9cb373" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37681 } } peers { permanent_uuid: "34c9ac0ce67d4df5863681360f27c4ac" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 37827 } }
I20260811 06:30:30.176925 10115 leader_election.cc:288] T 30e10a0693144530ab60b8070ff243cc P 57b2448bb01c461a964ee8ceca7f2315 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2e276abb6ae14f2aab047be5cb9cb373 (127.28.239.194:37681), 34c9ac0ce67d4df5863681360f27c4ac (127.28.239.195:37827)
I20260811 06:30:30.177623 9939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30e10a0693144530ab60b8070ff243cc" candidate_uuid: "57b2448bb01c461a964ee8ceca7f2315" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34c9ac0ce67d4df5863681360f27c4ac" is_pre_election: true
I20260811 06:30:30.177601 9744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30e10a0693144530ab60b8070ff243cc" candidate_uuid: "57b2448bb01c461a964ee8ceca7f2315" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e276abb6ae14f2aab047be5cb9cb373" is_pre_election: true
I20260811 06:30:30.177894 9744 raft_consensus.cc:2454] T 30e10a0693144530ab60b8070ff243cc P 2e276abb6ae14f2aab047be5cb9cb373 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57b2448bb01c461a964ee8ceca7f2315 in term 0.
I20260811 06:30:30.177911 9939 raft_consensus.cc:2454] T 30e10a0693144530ab60b8070ff243cc P 34c9ac0ce67d4df5863681360f27c4ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57b2448bb01c461a964ee8ceca7f2315 in term 0.
I20260811 06:30:30.178467 9413 leader_election.cc:302] T 30e10a0693144530ab60b8070ff243cc P 57b2448bb01c461a964ee8ceca7f2315 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e276abb6ae14f2aab047be5cb9cb373, 57b2448bb01c461a964ee8ceca7f2315; no voters:
I20260811 06:30:30.178828 10115 raft_consensus.cc:2781] T 30e10a0693144530ab60b8070ff243cc P 57b2448bb01c461a964ee8ceca7f2315 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:30:30.178972 10115 raft_consensus.cc:482] T 30e10a0693144530ab60b8070ff243cc P 57b2448bb01c461a964ee8ceca7f2315 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:30:30.179080 10115 raft_consensus.cc:3037] T 30e10a0693144530ab60b8070ff243cc P 57b2448bb01c461a964ee8ceca7f2315 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:30:30.182632 10115 raft_consensus.cc:504] T 30e10a0693144530ab60b8070ff243cc P 57b2448bb01c461a964ee8ceca7f2315 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b2448bb01c461a964ee8ceca7f2315" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 42829 } } peers { permanent_uuid: "2e276abb6ae14f2aab047be5cb9cb373" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37681 } } peers { permanent_uuid: "34c9ac0ce67d4df5863681360f27c4ac" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 37827 } }
I20260811 06:30:30.183363 10115 leader_election.cc:288] T 30e10a0693144530ab60b8070ff243cc P 57b2448bb01c461a964ee8ceca7f2315 [CANDIDATE]: Term 1 election: Requested vote from peers 2e276abb6ae14f2aab047be5cb9cb373 (127.28.239.194:37681), 34c9ac0ce67d4df5863681360f27c4ac (127.28.239.195:37827)
I20260811 06:30:30.184089 9939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30e10a0693144530ab60b8070ff243cc" candidate_uuid: "57b2448bb01c461a964ee8ceca7f2315" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34c9ac0ce67d4df5863681360f27c4ac"
I20260811 06:30:30.184089 9744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30e10a0693144530ab60b8070ff243cc" candidate_uuid: "57b2448bb01c461a964ee8ceca7f2315" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e276abb6ae14f2aab047be5cb9cb373"
I20260811 06:30:30.184332 9744 raft_consensus.cc:3037] T 30e10a0693144530ab60b8070ff243cc P 2e276abb6ae14f2aab047be5cb9cb373 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:30:30.184350 9939 raft_consensus.cc:3037] T 30e10a0693144530ab60b8070ff243cc P 34c9ac0ce67d4df5863681360f27c4ac [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:30:30.186785 9939 raft_consensus.cc:2454] T 30e10a0693144530ab60b8070ff243cc P 34c9ac0ce67d4df5863681360f27c4ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57b2448bb01c461a964ee8ceca7f2315 in term 1.
I20260811 06:30:30.187364 9411 leader_election.cc:302] T 30e10a0693144530ab60b8070ff243cc P 57b2448bb01c461a964ee8ceca7f2315 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34c9ac0ce67d4df5863681360f27c4ac, 57b2448bb01c461a964ee8ceca7f2315; no voters:
I20260811 06:30:30.187408 9744 raft_consensus.cc:2454] T 30e10a0693144530ab60b8070ff243cc P 2e276abb6ae14f2aab047be5cb9cb373 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57b2448bb01c461a964ee8ceca7f2315 in term 1.
I20260811 06:30:30.187724 10115 raft_consensus.cc:2781] T 30e10a0693144530ab60b8070ff243cc P 57b2448bb01c461a964ee8ceca7f2315 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:30:30.187994 10115 raft_consensus.cc:686] T 30e10a0693144530ab60b8070ff243cc P 57b2448bb01c461a964ee8ceca7f2315 [term 1 LEADER]: Becoming Leader. State: Replica: 57b2448bb01c461a964ee8ceca7f2315, State: Running, Role: LEADER
I20260811 06:30:30.188356 10115 consensus_queue.cc:237] T 30e10a0693144530ab60b8070ff243cc P 57b2448bb01c461a964ee8ceca7f2315 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b2448bb01c461a964ee8ceca7f2315" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 42829 } } peers { permanent_uuid: "2e276abb6ae14f2aab047be5cb9cb373" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37681 } } peers { permanent_uuid: "34c9ac0ce67d4df5863681360f27c4ac" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 37827 } }
I20260811 06:30:30.191386 9267 catalog_manager.cc:5495] T 30e10a0693144530ab60b8070ff243cc P 57b2448bb01c461a964ee8ceca7f2315 reported cstate change: term changed from 0 to 1, leader changed from <none> to 57b2448bb01c461a964ee8ceca7f2315 (127.28.239.193). New cstate: current_term: 1 leader_uuid: "57b2448bb01c461a964ee8ceca7f2315" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57b2448bb01c461a964ee8ceca7f2315" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 42829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2e276abb6ae14f2aab047be5cb9cb373" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34c9ac0ce67d4df5863681360f27c4ac" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 37827 } health_report { overall_health: UNKNOWN } } }
I20260811 06:30:30.242395 9744 raft_consensus.cc:1261] T 30e10a0693144530ab60b8070ff243cc P 2e276abb6ae14f2aab047be5cb9cb373 [term 1 FOLLOWER]: Refusing update from remote peer 57b2448bb01c461a964ee8ceca7f2315: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:30:30.242476 9939 raft_consensus.cc:1261] T 30e10a0693144530ab60b8070ff243cc P 34c9ac0ce67d4df5863681360f27c4ac [term 1 FOLLOWER]: Refusing update from remote peer 57b2448bb01c461a964ee8ceca7f2315: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:30:30.242941 10115 consensus_queue.cc:1035] T 30e10a0693144530ab60b8070ff243cc P 57b2448bb01c461a964ee8ceca7f2315 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e276abb6ae14f2aab047be5cb9cb373" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:30:30.243239 10060 consensus_queue.cc:1035] T 30e10a0693144530ab60b8070ff243cc P 57b2448bb01c461a964ee8ceca7f2315 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34c9ac0ce67d4df5863681360f27c4ac" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 37827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:30:30.304457 9267 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33954:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:30:30.304673 9267 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33954:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:30:30.307772 9267 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2678652809584c6cb6c707b112de9a62: Sending DeleteTablet for 3 replicas of tablet 30e10a0693144530ab60b8070ff243cc
I20260811 06:30:30.308911 9705 tablet_service.cc:1514] Processing DeleteTablet for tablet 30e10a0693144530ab60b8070ff243cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:30:30 UTC) from {username='oryx', principal='oryx/127.28.239.254@KRBTEST.COM'} at 127.0.0.1:42208
I20260811 06:30:30.309386 9490 tablet_service.cc:1514] Processing DeleteTablet for tablet 30e10a0693144530ab60b8070ff243cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:30:30 UTC) from {username='oryx', principal='oryx/127.28.239.254@KRBTEST.COM'} at 127.0.0.1:51296
I20260811 06:30:30.309424 10086 tablet_replica.cc:330] T 30e10a0693144530ab60b8070ff243cc P 2e276abb6ae14f2aab047be5cb9cb373: stopping tablet replica
I20260811 06:30:30.309742 10085 tablet_replica.cc:330] T 30e10a0693144530ab60b8070ff243cc P 57b2448bb01c461a964ee8ceca7f2315: stopping tablet replica
I20260811 06:30:30.309724 10086 raft_consensus.cc:2229] T 30e10a0693144530ab60b8070ff243cc P 2e276abb6ae14f2aab047be5cb9cb373 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:30:30.309942 10086 raft_consensus.cc:2258] T 30e10a0693144530ab60b8070ff243cc P 2e276abb6ae14f2aab047be5cb9cb373 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:30:30.309962 10085 raft_consensus.cc:2229] T 30e10a0693144530ab60b8070ff243cc P 57b2448bb01c461a964ee8ceca7f2315 [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:30:30.310304 10085 raft_consensus.cc:2258] T 30e10a0693144530ab60b8070ff243cc P 57b2448bb01c461a964ee8ceca7f2315 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:30:30.310648 9910 tablet_service.cc:1514] Processing DeleteTablet for tablet 30e10a0693144530ab60b8070ff243cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:30:30 UTC) from {username='oryx', principal='oryx/127.28.239.254@KRBTEST.COM'} at 127.0.0.1:42686
I20260811 06:30:30.311017 10084 tablet_replica.cc:330] T 30e10a0693144530ab60b8070ff243cc P 34c9ac0ce67d4df5863681360f27c4ac: stopping tablet replica
I20260811 06:30:30.311285 10084 raft_consensus.cc:2229] T 30e10a0693144530ab60b8070ff243cc P 34c9ac0ce67d4df5863681360f27c4ac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:30:30.311492 10084 raft_consensus.cc:2258] T 30e10a0693144530ab60b8070ff243cc P 34c9ac0ce67d4df5863681360f27c4ac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:30:30.312251 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 9381
I20260811 06:30:30.312444 10085 ts_tablet_manager.cc:1905] T 30e10a0693144530ab60b8070ff243cc P 57b2448bb01c461a964ee8ceca7f2315: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:30:30.315629 10084 ts_tablet_manager.cc:1905] T 30e10a0693144530ab60b8070ff243cc P 34c9ac0ce67d4df5863681360f27c4ac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:30:30.320403 10085 ts_tablet_manager.cc:1918] T 30e10a0693144530ab60b8070ff243cc P 57b2448bb01c461a964ee8ceca7f2315: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 06:30:30.320570 10085 log.cc:1196] T 30e10a0693144530ab60b8070ff243cc P 57b2448bb01c461a964ee8ceca7f2315: Deleting WAL directory at /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/minicluster-data/ts-0/wal/wals/30e10a0693144530ab60b8070ff243cc
I20260811 06:30:30.321054 10085 ts_tablet_manager.cc:1939] T 30e10a0693144530ab60b8070ff243cc P 57b2448bb01c461a964ee8ceca7f2315: Deleting consensus metadata
I20260811 06:30:30.322528 10086 ts_tablet_manager.cc:1905] T 30e10a0693144530ab60b8070ff243cc P 2e276abb6ae14f2aab047be5cb9cb373: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 11 06:30:30 dist-test-slave-gkw7 krb5kdc[9201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429829, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.28.239.193@KRBTEST.COM
I20260811 06:30:30.330090 10086 ts_tablet_manager.cc:1918] T 30e10a0693144530ab60b8070ff243cc P 2e276abb6ae14f2aab047be5cb9cb373: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 06:30:30.330272 10086 log.cc:1196] T 30e10a0693144530ab60b8070ff243cc P 2e276abb6ae14f2aab047be5cb9cb373: Deleting WAL directory at /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/minicluster-data/ts-1/wal/wals/30e10a0693144530ab60b8070ff243cc
I20260811 06:30:30.331892 9244 catalog_manager.cc:4841] TS 57b2448bb01c461a964ee8ceca7f2315 (127.28.239.193:42829): tablet 30e10a0693144530ab60b8070ff243cc (table test-table [id=c2cc565794914868a96b9e02d1ab1229]) successfully deleted
I20260811 06:30:30.332362 10086 ts_tablet_manager.cc:1939] T 30e10a0693144530ab60b8070ff243cc P 2e276abb6ae14f2aab047be5cb9cb373: Deleting consensus metadata
I20260811 06:30:30.332545 10084 ts_tablet_manager.cc:1918] T 30e10a0693144530ab60b8070ff243cc P 34c9ac0ce67d4df5863681360f27c4ac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 06:30:30.332794 10084 log.cc:1196] T 30e10a0693144530ab60b8070ff243cc P 34c9ac0ce67d4df5863681360f27c4ac: Deleting WAL directory at /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786429755927508-29631-0/minicluster-data/ts-2/wal/wals/30e10a0693144530ab60b8070ff243cc
I20260811 06:30:30.333326 10084 ts_tablet_manager.cc:1939] T 30e10a0693144530ab60b8070ff243cc P 34c9ac0ce67d4df5863681360f27c4ac: Deleting consensus metadata
I20260811 06:30:30.334654 9242 catalog_manager.cc:4841] TS 34c9ac0ce67d4df5863681360f27c4ac (127.28.239.195:37827): tablet 30e10a0693144530ab60b8070ff243cc (table test-table [id=c2cc565794914868a96b9e02d1ab1229]) successfully deleted
I20260811 06:30:30.336647 10045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:30.313696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51346 (local address 127.28.239.193:42829)
0811 06:30:30.313877 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:30.313883 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:30.315525 (+ 1642us) server_negotiation.cc:407] Connection header received
0811 06:30:30.315860 (+ 335us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:30.315865 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:30.315945 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:30.316019 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:30.317870 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:30.319147 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:30.321070 (+ 1923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:30.321854 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:30.321971 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:30.327705 (+ 5734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:30.327725 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:30.327749 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:30:30.327806 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:30.332487 (+ 4681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:30.333627 (+ 1140us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:30.333641 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:30.333648 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:30.333814 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:30.334478 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:30.334482 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:30.334485 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:30.335086 (+ 601us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:30.335420 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:30.335996 (+ 576us) server_negotiation.cc:299] Negotiation successful
0811 06:30:30.336243 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 06:30:30.337460 9597 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:30:30.337913 9244 catalog_manager.cc:4841] TS 2e276abb6ae14f2aab047be5cb9cb373 (127.28.239.194:37681): tablet 30e10a0693144530ab60b8070ff243cc (table test-table [id=c2cc565794914868a96b9e02d1ab1229]) successfully deleted
I20260811 06:30:30.496111 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 9381
I20260811 06:30:30.529412 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 9613
Aug 11 06:30:30 dist-test-slave-gkw7 krb5kdc[9201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429829, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.28.239.194@KRBTEST.COM
I20260811 06:30:30.546003 10181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:30.530272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42212 (local address 127.28.239.194:37681)
0811 06:30:30.530765 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:30.530770 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:30.530786 (+ 16us) server_negotiation.cc:407] Connection header received
0811 06:30:30.530860 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:30.530864 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:30.530925 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:30.531016 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:30.532380 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:30.533147 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:30.534748 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:30.535288 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:30.535417 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:30.539752 (+ 4335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:30.539769 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:30.539788 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:30:30.539840 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:30.543175 (+ 3335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:30.543892 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:30.543901 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:30.543903 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:30.544068 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:30.544421 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:30.544425 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:30.544428 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:30.544855 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:30.545124 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:30.545425 (+ 301us) server_negotiation.cc:299] Negotiation successful
0811 06:30:30.545705 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":265,"threads_started":1}
I20260811 06:30:30.546803 9807 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:30:30.647080 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 9613
I20260811 06:30:30.675359 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 9816
Aug 11 06:30:30 dist-test-slave-gkw7 krb5kdc[9201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429829, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.28.239.195@KRBTEST.COM
I20260811 06:30:30.691308 10185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:30.676211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42700 (local address 127.28.239.195:37827)
0811 06:30:30.676712 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:30.676717 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:30.676733 (+ 16us) server_negotiation.cc:407] Connection header received
0811 06:30:30.676793 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:30.676797 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:30.676859 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:30.676962 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:30.678450 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:30.679228 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:30.680500 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:30.680938 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:30.681042 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:30.685595 (+ 4553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:30.685613 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:30.685631 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:30:30.685678 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:30.688463 (+ 2785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:30.689160 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:30.689193 (+ 33us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:30.689195 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:30.689326 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:30.689776 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:30.689780 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:30.689782 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:30.690191 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:30.690430 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:30.690789 (+ 359us) server_negotiation.cc:299] Negotiation successful
0811 06:30:30.691000 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":266,"threads_started":1}
I20260811 06:30:30.692054 10007 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:30:30.818845 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 9816
I20260811 06:30:30.849944 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 9220
I20260811 06:30:30.851430 9325 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:30:30.972524 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 9220
2026-08-11T06:30:30Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (6127 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786429755927508-29631-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:30:31 dist-test-slave-gkw7 krb5kdc[10191](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:30:31 dist-test-slave-gkw7 krb5kdc[10191](info): set up 2 sockets
Aug 11 06:30:31 dist-test-slave-gkw7 krb5kdc[10191](info): commencing operation
krb5kdc: starting...
W20260811 06:30:32.358126 29631 mini_kdc.cc:121] Time spent starting KDC: real 1.331s user 0.001s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786429755927508-29631-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786429755927508-29631-0/krb5kdc/test-user.keytab.
Aug 11 06:30:32 dist-test-slave-gkw7 krb5kdc[10191](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429832, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:30:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:30:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.239.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.239.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.239.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab.
Entry for principal kudu/127.28.239.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab.
I20260811 06:30:32.645007 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786429755927508-29631-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786429755927508-29631-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786429755927508-29631-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:40903
--webserver_interface=127.28.239.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab
--principal=kudu/127.28.239.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:40889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.239.254:40903
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:30:32.997900 10208 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:30:32.998216 10208 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:30:32.998394 10208 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:30:33.005741 10208 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:30:33.005858 10208 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:30:33.005941 10208 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:30:33.006048 10208 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:30:33.006145 10208 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:30:33.016517 10208 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:40889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786429755927508-29631-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786429755927508-29631-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.239.254:40903
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:40903
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.239.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786429755927508-29631-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:30:33.017956 10208 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:30:33.019349 10208 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:30:33.031836 10217 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:30:33.031922 10215 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:30:33.031922 10221 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:30:33.033187 10208 server_base.cc:1034] running on GCE node
I20260811 06:30:33.034096 10208 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:30:33.035969 10208 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:30:33.037250 10208 hybrid_clock.cc:648] HybridClock initialized: now 1786429833037201 us; error 39 us; skew 500 ppm
I20260811 06:30:33.037721 10208 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-11T06:30:33Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (2138 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:30:33 dist-test-slave-gkw7 krb5kdc[10228](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:30:33 dist-test-slave-gkw7 krb5kdc[10228](info): set up 2 sockets
Aug 11 06:30:33 dist-test-slave-gkw7 krb5kdc[10228](info): commencing operation
krb5kdc: starting...
W20260811 06:30:34.467547 29631 mini_kdc.cc:121] Time spent starting KDC: real 1.302s user 0.001s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/krb5kdc/test-user.keytab.
Aug 11 06:30:34 dist-test-slave-gkw7 krb5kdc[10228](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429834, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:30:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:30:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.239.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.239.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.239.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab.
Entry for principal kudu/127.28.239.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab.
I20260811 06:30:34.750145 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:35823
--webserver_interface=127.28.239.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab
--principal=kudu/127.28.239.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:42763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.239.254:35823
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:30:35.097321 10247 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:30:35.097654 10247 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:30:35.097759 10247 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:30:35.105057 10247 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:30:35.105170 10247 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:30:35.105218 10247 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:30:35.105279 10247 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:30:35.105324 10247 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:30:35.115401 10247 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:42763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.239.254:35823
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab
--principal=kudu/127.28.239.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:35823
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.239.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:30:35.116894 10247 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:30:35.118348 10247 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:30:35.131312 10255 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:30:35.131379 10257 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:30:35.131414 10252 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:30:35.132429 10247 server_base.cc:1034] running on GCE node
I20260811 06:30:35.133271 10247 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:30:35.134874 10247 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:30:35.136117 10247 hybrid_clock.cc:648] HybridClock initialized: now 1786429835136068 us; error 51 us; skew 500 ppm
I20260811 06:30:35.136520 10247 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:30:35 dist-test-slave-gkw7 krb5kdc[10228](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429835, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:30:35.140045 10247 init.cc:377] Logged in from keytab as kudu/127.28.239.254@KRBTEST.COM (short username kudu)
I20260811 06:30:35.141722 10247 webserver.cc:460] Webserver started at http://127.28.239.254:39475/ using document root <none> and password file <none>
I20260811 06:30:35.142480 10247 fs_manager.cc:359] Metadata directory not provided
I20260811 06:30:35.142611 10247 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:30:35.142962 10247 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:30:35.146191 10247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/master-0/data/instance:
uuid: "f9b54697de114ef486497b5f41b0d437"
format_stamp: "Formatted at 2026-08-11 06:30:35 on dist-test-slave-gkw7"
server_key: "1905875145f45d68e63839db9086fad7"
server_key_iv: "bac95d0fa3d8f6b7b35ff913f51ea1da"
server_key_version: "encryptionkey@0"
I20260811 06:30:35.147012 10247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/master-0/wal/instance:
uuid: "f9b54697de114ef486497b5f41b0d437"
format_stamp: "Formatted at 2026-08-11 06:30:35 on dist-test-slave-gkw7"
server_key: "1905875145f45d68e63839db9086fad7"
server_key_iv: "bac95d0fa3d8f6b7b35ff913f51ea1da"
server_key_version: "encryptionkey@0"
I20260811 06:30:35.152400 10247 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 06:30:35.156206 10265 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:35.157953 10247 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:30:35.158142 10247 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/master-0/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/master-0/wal
uuid: "f9b54697de114ef486497b5f41b0d437"
format_stamp: "Formatted at 2026-08-11 06:30:35 on dist-test-slave-gkw7"
server_key: "1905875145f45d68e63839db9086fad7"
server_key_iv: "bac95d0fa3d8f6b7b35ff913f51ea1da"
server_key_version: "encryptionkey@0"
I20260811 06:30:35.158336 10247 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:30:35.173498 10247 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:30:35.177891 10247 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:30:35.178197 10247 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:30:35.198427 10247 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.254:35823
I20260811 06:30:35.198474 10344 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.254:35823 every 8 connection(s)
I20260811 06:30:35.199890 10247 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
I20260811 06:30:35.204140 10345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:30:35.207998 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 10247
I20260811 06:30:35.208246 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/master-0/wal/instance
I20260811 06:30:35.209194 29631 external_mini_cluster.cc:1428] Setting key 332fad7b6fde7742cc1213f1baacd0fd
I20260811 06:30:35.219100 10345 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9b54697de114ef486497b5f41b0d437: Bootstrap starting.
I20260811 06:30:35.223446 10345 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f9b54697de114ef486497b5f41b0d437: Neither blocks nor log segments found. Creating new log.
I20260811 06:30:35.224984 10345 log.cc:824] T 00000000000000000000000000000000 P f9b54697de114ef486497b5f41b0d437: Log is configured to *not* fsync() on all Append() calls
Aug 11 06:30:35 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429834, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:30:35.228533 10345 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9b54697de114ef486497b5f41b0d437: No bootstrap required, opened a new log
I20260811 06:30:35.234122 10348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:35.211565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36332 (local address 127.28.239.254:35823)
0811 06:30:35.212376 (+ 811us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:35.212389 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:35.212431 (+ 42us) server_negotiation.cc:407] Connection header received
0811 06:30:35.214211 (+ 1780us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:35.214237 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:35.214639 (+ 402us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:35.214983 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:35.217269 (+ 2286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:35.218648 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:35.220582 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:35.221267 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:35.221411 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:35.226535 (+ 5124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:35.226544 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:35.226559 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:30:35.226596 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:35.229882 (+ 3286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:35.230603 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:35.230611 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:35.230618 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:35.230805 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:35.231206 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:35.231221 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:35.231223 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:35.231782 (+ 559us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:35.232121 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:35.232573 (+ 452us) server_negotiation.cc:299] Negotiation successful
0811 06:30:35.233015 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":603,"thread_start_us":391,"threads_started":1}
I20260811 06:30:35.237510 10345 raft_consensus.cc:348] T 00000000000000000000000000000000 P f9b54697de114ef486497b5f41b0d437 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9b54697de114ef486497b5f41b0d437" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 35823 } }
I20260811 06:30:35.237991 10345 raft_consensus.cc:374] T 00000000000000000000000000000000 P f9b54697de114ef486497b5f41b0d437 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:30:35.238142 10345 raft_consensus.cc:729] T 00000000000000000000000000000000 P f9b54697de114ef486497b5f41b0d437 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9b54697de114ef486497b5f41b0d437, State: Initialized, Role: FOLLOWER
I20260811 06:30:35.238737 10345 consensus_queue.cc:260] T 00000000000000000000000000000000 P f9b54697de114ef486497b5f41b0d437 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9b54697de114ef486497b5f41b0d437" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 35823 } }
I20260811 06:30:35.239012 10345 raft_consensus.cc:388] T 00000000000000000000000000000000 P f9b54697de114ef486497b5f41b0d437 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:30:35.239182 10345 raft_consensus.cc:482] T 00000000000000000000000000000000 P f9b54697de114ef486497b5f41b0d437 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:30:35.239382 10345 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f9b54697de114ef486497b5f41b0d437 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:30:35.242157 10345 raft_consensus.cc:504] T 00000000000000000000000000000000 P f9b54697de114ef486497b5f41b0d437 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9b54697de114ef486497b5f41b0d437" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 35823 } }
I20260811 06:30:35.242821 10345 leader_election.cc:302] T 00000000000000000000000000000000 P f9b54697de114ef486497b5f41b0d437 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f9b54697de114ef486497b5f41b0d437; no voters:
I20260811 06:30:35.243508 10345 leader_election.cc:288] T 00000000000000000000000000000000 P f9b54697de114ef486497b5f41b0d437 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:30:35.243691 10350 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f9b54697de114ef486497b5f41b0d437 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:30:35.244541 10350 raft_consensus.cc:686] T 00000000000000000000000000000000 P f9b54697de114ef486497b5f41b0d437 [term 1 LEADER]: Becoming Leader. State: Replica: f9b54697de114ef486497b5f41b0d437, State: Running, Role: LEADER
I20260811 06:30:35.245194 10350 consensus_queue.cc:237] T 00000000000000000000000000000000 P f9b54697de114ef486497b5f41b0d437 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9b54697de114ef486497b5f41b0d437" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 35823 } }
I20260811 06:30:35.245972 10345 sys_catalog.cc:564] T 00000000000000000000000000000000 P f9b54697de114ef486497b5f41b0d437 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:30:35.248647 10354 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9b54697de114ef486497b5f41b0d437 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f9b54697de114ef486497b5f41b0d437. Latest consensus state: current_term: 1 leader_uuid: "f9b54697de114ef486497b5f41b0d437" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9b54697de114ef486497b5f41b0d437" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 35823 } } }
I20260811 06:30:35.249143 10354 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9b54697de114ef486497b5f41b0d437 [sys.catalog]: This master's current role is: LEADER
I20260811 06:30:35.249361 10351 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9b54697de114ef486497b5f41b0d437 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f9b54697de114ef486497b5f41b0d437" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9b54697de114ef486497b5f41b0d437" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 35823 } } }
I20260811 06:30:35.249782 10351 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9b54697de114ef486497b5f41b0d437 [sys.catalog]: This master's current role is: LEADER
I20260811 06:30:35.253793 10367 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:30:35.259123 10367 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:30:35.268503 10367 catalog_manager.cc:1347] Generated new cluster ID: 227ea0dd9739483084e87e24c6b415b2
I20260811 06:30:35.268639 10367 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:30:35.285382 10367 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:30:35.286897 10367 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:30:35.296434 10367 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f9b54697de114ef486497b5f41b0d437: Generated new TSK 0
I20260811 06:30:35.297346 10367 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.239.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.239.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.239.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.193.keytab.
Entry for principal kudu/127.28.239.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.193.keytab.
I20260811 06:30:35.377822 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.193:0
--local_ip_for_outbound_sockets=127.28.239.193
--webserver_interface=127.28.239.193
--webserver_port=0
--tserver_master_addrs=127.28.239.254:35823
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.193.keytab
--principal=kudu/127.28.239.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:42763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:30:35.737844 10389 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:30:35.738199 10389 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:30:35.738363 10389 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:30:35.746202 10389 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:30:35.746351 10389 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:30:35.746555 10389 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.193
I20260811 06:30:35.756973 10389 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:42763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.193.keytab
--principal=kudu/127.28.239.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.239.193
--webserver_port=0
--tserver_master_addrs=127.28.239.254:35823
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.193
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:30:35.758414 10389 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:30:35.759764 10389 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:30:35.774101 10402 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:30:35.774214 10399 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:30:35.774119 10397 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:30:35.783648 10389 server_base.cc:1034] running on GCE node
I20260811 06:30:35.784487 10389 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:30:35.785815 10389 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:30:35.787110 10389 hybrid_clock.cc:648] HybridClock initialized: now 1786429835787025 us; error 73 us; skew 500 ppm
I20260811 06:30:35.787473 10389 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:30:35 dist-test-slave-gkw7 krb5kdc[10228](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429835, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:30:35.790943 10389 init.cc:377] Logged in from keytab as kudu/127.28.239.193@KRBTEST.COM (short username kudu)
I20260811 06:30:35.792554 10389 webserver.cc:460] Webserver started at http://127.28.239.193:37851/ using document root <none> and password file <none>
I20260811 06:30:35.793315 10389 fs_manager.cc:359] Metadata directory not provided
I20260811 06:30:35.793465 10389 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:30:35.793833 10389 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:30:35.797005 10389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/ts-0/data/instance:
uuid: "ad25e7e7049648fb9c3a28fd6cef9d5d"
format_stamp: "Formatted at 2026-08-11 06:30:35 on dist-test-slave-gkw7"
server_key: "a28c363740db6c6f722a790d83699598"
server_key_iv: "3fe06c78ef7663e6df19aaa96e5a073c"
server_key_version: "encryptionkey@0"
I20260811 06:30:35.797896 10389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/ts-0/wal/instance:
uuid: "ad25e7e7049648fb9c3a28fd6cef9d5d"
format_stamp: "Formatted at 2026-08-11 06:30:35 on dist-test-slave-gkw7"
server_key: "a28c363740db6c6f722a790d83699598"
server_key_iv: "3fe06c78ef7663e6df19aaa96e5a073c"
server_key_version: "encryptionkey@0"
I20260811 06:30:35.803181 10389 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 06:30:35.806768 10412 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:35.808094 10389 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 06:30:35.808276 10389 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/ts-0/wal
uuid: "ad25e7e7049648fb9c3a28fd6cef9d5d"
format_stamp: "Formatted at 2026-08-11 06:30:35 on dist-test-slave-gkw7"
server_key: "a28c363740db6c6f722a790d83699598"
server_key_iv: "3fe06c78ef7663e6df19aaa96e5a073c"
server_key_version: "encryptionkey@0"
I20260811 06:30:35.808487 10389 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:30:35.824308 10389 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:30:35.828835 10389 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:30:35.829151 10389 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:30:35.830291 10389 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:30:35.831749 10389 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:30:35.831853 10389 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:35.832010 10389 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:30:35.832105 10389 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:35.867151 10389 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.193:43093
I20260811 06:30:35.867190 10585 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.193:43093 every 8 connection(s)
I20260811 06:30:35.868535 10389 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
I20260811 06:30:35.868952 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 10389
I20260811 06:30:35.869166 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/ts-0/wal/instance
I20260811 06:30:35.870172 29631 external_mini_cluster.cc:1428] Setting key 88a61c1d6af1464558005327a943bfb2
Aug 11 06:30:35 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429835, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.193@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:30:35.891242 10588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:35.871358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.193:54733 (local address 127.28.239.254:35823)
0811 06:30:35.871854 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:35.871858 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:35.873024 (+ 1166us) server_negotiation.cc:407] Connection header received
0811 06:30:35.875025 (+ 2001us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:35.875031 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:35.875116 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:35.875256 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:35.876840 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:35.877669 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:35.879262 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:35.879679 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:35.879822 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:35.883642 (+ 3820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:35.883650 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:35.883656 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:30:35.883700 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:35.887475 (+ 3775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:35.888371 (+ 896us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:35.888381 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:35.888387 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:35.888575 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:35.889059 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:35.889068 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:35.889074 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:35.889481 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:35.889670 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:35.890477 (+ 807us) server_negotiation.cc:299] Negotiation successful
0811 06:30:35.890742 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":263,"threads_started":1}
I20260811 06:30:35.892005 10590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:35.872054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:35823 (local address 127.28.239.193:54733)
0811 06:30:35.872870 (+ 816us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:35.872909 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:35.874678 (+ 1769us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:35.875500 (+ 822us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:35.875508 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:35.876041 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:35.876624 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:35.876641 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:35.877823 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:35.877829 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:35.879103 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:35.879116 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:35.879916 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:35.879927 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:35.880183 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:35.880892 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:30:35.880915 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:35.883393 (+ 2478us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:30:35.887672 (+ 4279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:30:35.887680 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:30:35.887694 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:30:35.888219 (+ 525us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:30:35.888706 (+ 487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:30:35.888710 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:30:35.888713 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:30:35.888933 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:30:35.889664 (+ 731us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:30:35.889672 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:30:35.890043 (+ 371us) client_negotiation.cc:769] Sending connection context
0811 06:30:35.890331 (+ 288us) client_negotiation.cc:240] Negotiation successful
0811 06:30:35.890778 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":327,"threads_started":1}
I20260811 06:30:35.893908 10586 heartbeater.cc:347] Connected to a master server at 127.28.239.254:35823
I20260811 06:30:35.894258 10586 heartbeater.cc:464] Registering TS with master...
I20260811 06:30:35.895056 10586 heartbeater.cc:511] Master 127.28.239.254:35823 requested a full tablet report, sending...
I20260811 06:30:35.897405 10295 ts_manager.cc:194] Registered new tserver with Master: ad25e7e7049648fb9c3a28fd6cef9d5d (127.28.239.193:43093)
I20260811 06:30:35.898964 10295 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.239.193@KRBTEST.COM'} at 127.28.239.193:54733
WARNING: no policy specified for kudu/127.28.239.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.239.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.239.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.194.keytab.
Entry for principal kudu/127.28.239.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.194.keytab.
I20260811 06:30:35.933192 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.194:0
--local_ip_for_outbound_sockets=127.28.239.194
--webserver_interface=127.28.239.194
--webserver_port=0
--tserver_master_addrs=127.28.239.254:35823
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.194.keytab
--principal=kudu/127.28.239.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:42763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:30:36.298692 10594 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:30:36.299086 10594 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:30:36.299199 10594 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:30:36.307195 10594 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:30:36.307335 10594 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:30:36.307461 10594 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.194
I20260811 06:30:36.318403 10594 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:42763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.194.keytab
--principal=kudu/127.28.239.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.239.194
--webserver_port=0
--tserver_master_addrs=127.28.239.254:35823
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.194
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:30:36.319782 10594 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:30:36.321190 10594 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:30:36.336028 10601 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:30:36.336031 10602 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:30:36.336812 10604 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:30:36.337008 10594 server_base.cc:1034] running on GCE node
I20260811 06:30:36.337883 10594 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:30:36.338878 10594 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:30:36.340148 10594 hybrid_clock.cc:648] HybridClock initialized: now 1786429836340043 us; error 48 us; skew 500 ppm
I20260811 06:30:36.340461 10594 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:30:36 dist-test-slave-gkw7 krb5kdc[10228](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429836, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:30:36.343998 10594 init.cc:377] Logged in from keytab as kudu/127.28.239.194@KRBTEST.COM (short username kudu)
I20260811 06:30:36.345741 10594 webserver.cc:460] Webserver started at http://127.28.239.194:44739/ using document root <none> and password file <none>
I20260811 06:30:36.346624 10594 fs_manager.cc:359] Metadata directory not provided
I20260811 06:30:36.346768 10594 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:30:36.347129 10594 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:30:36.350567 10594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/ts-1/data/instance:
uuid: "f283822b11c34681844bde87d93c30d1"
format_stamp: "Formatted at 2026-08-11 06:30:36 on dist-test-slave-gkw7"
server_key: "959ba17aebe2902b3777ab49d034c1b8"
server_key_iv: "ec9920d68de8f24cd4ed02aa0028e28c"
server_key_version: "encryptionkey@0"
I20260811 06:30:36.351459 10594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/ts-1/wal/instance:
uuid: "f283822b11c34681844bde87d93c30d1"
format_stamp: "Formatted at 2026-08-11 06:30:36 on dist-test-slave-gkw7"
server_key: "959ba17aebe2902b3777ab49d034c1b8"
server_key_iv: "ec9920d68de8f24cd4ed02aa0028e28c"
server_key_version: "encryptionkey@0"
I20260811 06:30:36.357182 10594 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260811 06:30:36.361210 10619 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:36.362974 10594 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:30:36.363170 10594 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/ts-1/wal
uuid: "f283822b11c34681844bde87d93c30d1"
format_stamp: "Formatted at 2026-08-11 06:30:36 on dist-test-slave-gkw7"
server_key: "959ba17aebe2902b3777ab49d034c1b8"
server_key_iv: "ec9920d68de8f24cd4ed02aa0028e28c"
server_key_version: "encryptionkey@0"
I20260811 06:30:36.363371 10594 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:30:36.387643 10594 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:30:36.392392 10594 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:30:36.392755 10594 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:30:36.394037 10594 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:30:36.395634 10594 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:30:36.395779 10594 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:36.395973 10594 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:30:36.396060 10594 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:36.434120 10594 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.194:44689
I20260811 06:30:36.434166 10781 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.194:44689 every 8 connection(s)
I20260811 06:30:36.435668 10594 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
I20260811 06:30:36.443389 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 10594
I20260811 06:30:36.443567 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/ts-1/wal/instance
I20260811 06:30:36.444080 29631 external_mini_cluster.cc:1428] Setting key bfb18b50c1c8ba011d5d8163fa1eeb92
Aug 11 06:30:36 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429836, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.194@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:30:36.458132 10786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:36.438645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.194:59545 (local address 127.28.239.254:35823)
0811 06:30:36.439130 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:36.439135 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:36.440288 (+ 1153us) server_negotiation.cc:407] Connection header received
0811 06:30:36.442370 (+ 2082us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:36.442376 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:36.442472 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:36.442610 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:36.444398 (+ 1788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:36.445287 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:36.447261 (+ 1974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:36.447784 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:36.447927 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:36.452031 (+ 4104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:36.452037 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:36.452041 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:30:36.452075 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:36.454793 (+ 2718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:36.455589 (+ 796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:36.455594 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:36.455596 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:36.455723 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:36.456189 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:36.456198 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:36.456200 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:36.456464 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:36.456636 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:36.457481 (+ 845us) server_negotiation.cc:299] Negotiation successful
0811 06:30:36.457753 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":242,"threads_started":1}
I20260811 06:30:36.459195 10788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:36.439276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:35823 (local address 127.28.239.194:59545)
0811 06:30:36.440096 (+ 820us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:36.440150 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:36.442059 (+ 1909us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:36.442858 (+ 799us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:36.442866 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:36.443440 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:36.444199 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:36.444228 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:36.445457 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:36.445464 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:36.447090 (+ 1626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:36.447104 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:36.448034 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:36.448042 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:36.448392 (+ 350us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:36.449171 (+ 779us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:30:36.449196 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:36.451789 (+ 2593us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:30:36.454946 (+ 3157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:30:36.454956 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:30:36.454974 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:30:36.455433 (+ 459us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:30:36.455844 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:30:36.455848 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:30:36.455852 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:30:36.456049 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:30:36.456603 (+ 554us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:30:36.456612 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:30:36.457045 (+ 433us) client_negotiation.cc:769] Sending connection context
0811 06:30:36.457369 (+ 324us) client_negotiation.cc:240] Negotiation successful
0811 06:30:36.457963 (+ 594us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":295,"threads_started":1}
I20260811 06:30:36.460760 10784 heartbeater.cc:347] Connected to a master server at 127.28.239.254:35823
I20260811 06:30:36.461115 10784 heartbeater.cc:464] Registering TS with master...
I20260811 06:30:36.462046 10784 heartbeater.cc:511] Master 127.28.239.254:35823 requested a full tablet report, sending...
I20260811 06:30:36.464092 10295 ts_manager.cc:194] Registered new tserver with Master: f283822b11c34681844bde87d93c30d1 (127.28.239.194:44689)
I20260811 06:30:36.465411 10295 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.239.194@KRBTEST.COM'} at 127.28.239.194:59545
WARNING: no policy specified for kudu/127.28.239.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.239.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.239.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.195.keytab.
Entry for principal kudu/127.28.239.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.195.keytab.
I20260811 06:30:36.508612 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.195:0
--local_ip_for_outbound_sockets=127.28.239.195
--webserver_interface=127.28.239.195
--webserver_port=0
--tserver_master_addrs=127.28.239.254:35823
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.195.keytab
--principal=kudu/127.28.239.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:42763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:30:36.859098 10792 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:30:36.859417 10792 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:30:36.859517 10792 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:30:36.866915 10792 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:30:36.867027 10792 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:30:36.867220 10792 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.195
I20260811 06:30:36.877344 10792 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:42763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.195.keytab
--principal=kudu/127.28.239.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.239.195
--webserver_port=0
--tserver_master_addrs=127.28.239.254:35823
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.195
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:30:36.878728 10792 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:30:36.880079 10792 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:30:36.893846 10798 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:30:36.893939 10799 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:30:36.894500 10801 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:30:36.894834 10792 server_base.cc:1034] running on GCE node
I20260811 06:30:36.895443 10792 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:30:36.896456 10792 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:30:36.897631 10792 hybrid_clock.cc:648] HybridClock initialized: now 1786429836897574 us; error 50 us; skew 500 ppm
I20260811 06:30:36.897971 10792 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:30:36 dist-test-slave-gkw7 krb5kdc[10228](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429836, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:30:36.901197 10792 init.cc:377] Logged in from keytab as kudu/127.28.239.195@KRBTEST.COM (short username kudu)
I20260811 06:30:36.902190 10586 heartbeater.cc:503] Master 127.28.239.254:35823 was elected leader, sending a full tablet report...
I20260811 06:30:36.903008 10792 webserver.cc:460] Webserver started at http://127.28.239.195:44917/ using document root <none> and password file <none>
I20260811 06:30:36.903759 10792 fs_manager.cc:359] Metadata directory not provided
I20260811 06:30:36.903908 10792 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:30:36.904259 10792 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:30:36.907389 10792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/ts-2/data/instance:
uuid: "04c5ac8768c84023a5457f0b2617ef25"
format_stamp: "Formatted at 2026-08-11 06:30:36 on dist-test-slave-gkw7"
server_key: "35e3694c2c4ccb73b7589e2a08544128"
server_key_iv: "c5d89d381af8fd7abd67dca2cc74b769"
server_key_version: "encryptionkey@0"
I20260811 06:30:36.908135 10792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/ts-2/wal/instance:
uuid: "04c5ac8768c84023a5457f0b2617ef25"
format_stamp: "Formatted at 2026-08-11 06:30:36 on dist-test-slave-gkw7"
server_key: "35e3694c2c4ccb73b7589e2a08544128"
server_key_iv: "c5d89d381af8fd7abd67dca2cc74b769"
server_key_version: "encryptionkey@0"
I20260811 06:30:36.913652 10792 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260811 06:30:36.917277 10809 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:36.918788 10792 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:30:36.918977 10792 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/ts-2/wal
uuid: "04c5ac8768c84023a5457f0b2617ef25"
format_stamp: "Formatted at 2026-08-11 06:30:36 on dist-test-slave-gkw7"
server_key: "35e3694c2c4ccb73b7589e2a08544128"
server_key_iv: "c5d89d381af8fd7abd67dca2cc74b769"
server_key_version: "encryptionkey@0"
I20260811 06:30:36.919189 10792 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:30:36.944224 10792 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:30:36.949165 10792 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:30:36.949600 10792 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:30:36.950690 10792 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:30:36.952178 10792 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:30:36.952288 10792 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:36.952422 10792 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:30:36.952509 10792 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:36.989394 10792 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.195:42883
I20260811 06:30:36.989456 10980 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.195:42883 every 8 connection(s)
I20260811 06:30:36.990757 10792 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
I20260811 06:30:36.998598 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 10792
I20260811 06:30:36.998782 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/ts-2/wal/instance
I20260811 06:30:36.999269 29631 external_mini_cluster.cc:1428] Setting key 1fc943660666e1599d72b400227e6b02
Aug 11 06:30:37 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429836, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.195@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:30:37.013456 10983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:36.993729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.195:49891 (local address 127.28.239.254:35823)
0811 06:30:36.994370 (+ 641us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:36.994375 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:36.995163 (+ 788us) server_negotiation.cc:407] Connection header received
0811 06:30:36.997221 (+ 2058us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:36.997228 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:36.997335 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:36.997472 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:36.999352 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:37.000364 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:37.002471 (+ 2107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:37.003023 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:37.003148 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:37.006880 (+ 3732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:37.006887 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:37.006890 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:30:37.006926 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:37.009729 (+ 2803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:37.010586 (+ 857us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:37.010597 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:37.010600 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:37.010739 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:37.011271 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:37.011276 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:37.011278 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:37.011576 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:37.011775 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:37.012710 (+ 935us) server_negotiation.cc:299] Negotiation successful
0811 06:30:37.012998 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":526,"thread_start_us":293,"threads_started":1}
I20260811 06:30:37.014786 10985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:36.994139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:35823 (local address 127.28.239.195:49891)
0811 06:30:36.994932 (+ 793us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:36.994973 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:36.996906 (+ 1933us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:36.997779 (+ 873us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:36.997788 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:36.998406 (+ 618us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:36.999155 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:36.999176 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:37.000555 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:37.000562 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:37.002298 (+ 1736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:37.002315 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:37.003232 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:37.003237 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:37.003450 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:37.004177 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:30:37.004201 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:37.006633 (+ 2432us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:30:37.009890 (+ 3257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:30:37.009901 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:30:37.009927 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:30:37.010424 (+ 497us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:30:37.010870 (+ 446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:30:37.010875 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:30:37.010879 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:30:37.011110 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:30:37.011744 (+ 634us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:30:37.011753 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:30:37.012284 (+ 531us) client_negotiation.cc:769] Sending connection context
0811 06:30:37.012592 (+ 308us) client_negotiation.cc:240] Negotiation successful
0811 06:30:37.013225 (+ 633us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":545,"thread_start_us":308,"threads_started":1}
I20260811 06:30:37.016413 10981 heartbeater.cc:347] Connected to a master server at 127.28.239.254:35823
I20260811 06:30:37.016762 10981 heartbeater.cc:464] Registering TS with master...
I20260811 06:30:37.017616 10981 heartbeater.cc:511] Master 127.28.239.254:35823 requested a full tablet report, sending...
I20260811 06:30:37.019088 10295 ts_manager.cc:194] Registered new tserver with Master: 04c5ac8768c84023a5457f0b2617ef25 (127.28.239.195:42883)
I20260811 06:30:37.019904 10295 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.239.195@KRBTEST.COM'} at 127.28.239.195:49891
I20260811 06:30:37.027952 29631 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.28.239.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.28.239.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.28.239.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/krb5kdc/oryx_127.28.239.193.keytab.
Entry for principal oryx/127.28.239.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/krb5kdc/oryx_127.28.239.193.keytab.
WARNING: no policy specified for oryx/127.28.239.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.28.239.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.28.239.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/krb5kdc/oryx_127.28.239.194.keytab.
Entry for principal oryx/127.28.239.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/krb5kdc/oryx_127.28.239.194.keytab.
WARNING: no policy specified for oryx/127.28.239.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.28.239.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.28.239.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/krb5kdc/oryx_127.28.239.195.keytab.
Entry for principal oryx/127.28.239.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/krb5kdc/oryx_127.28.239.195.keytab.
I20260811 06:30:37.209482 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 10389
Aug 11 06:30:37 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429834, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.239.193@KRBTEST.COM
I20260811 06:30:37.226508 10996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:37.210768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36006 (local address 127.28.239.193:43093)
0811 06:30:37.211480 (+ 712us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:37.211488 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:37.211516 (+ 28us) server_negotiation.cc:407] Connection header received
0811 06:30:37.211584 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:37.211591 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:37.211816 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:37.211962 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:37.213363 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:37.214479 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:37.215707 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:37.216311 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:37.216410 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:37.220579 (+ 4169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:37.220592 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:37.220609 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:30:37.220646 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:37.223861 (+ 3215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:37.224593 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:37.224600 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:37.224603 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:37.224760 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:37.225087 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:37.225090 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:37.225092 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:37.225468 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:37.225821 (+ 353us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:37.225990 (+ 169us) server_negotiation.cc:299] Negotiation successful
0811 06:30:37.226203 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":551,"thread_start_us":389,"threads_started":1}
I20260811 06:30:37.227864 10579 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:30:37.316239 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 10389
I20260811 06:30:37.341480 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 10594
Aug 11 06:30:37 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429834, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.239.194@KRBTEST.COM
I20260811 06:30:37.358393 11000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:37.342518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37988 (local address 127.28.239.194:44689)
0811 06:30:37.343092 (+ 574us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:37.343100 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:37.343126 (+ 26us) server_negotiation.cc:407] Connection header received
0811 06:30:37.343189 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:37.343195 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:37.343428 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:37.343558 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:37.344860 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:37.346037 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:37.347495 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:37.348030 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:37.348124 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:37.352613 (+ 4489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:37.352624 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:37.352640 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:30:37.352682 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:37.355715 (+ 3033us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:37.356368 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:37.356375 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:37.356378 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:37.356508 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:37.356831 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:37.356834 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:37.356836 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:37.357171 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:37.357398 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:37.357756 (+ 358us) server_negotiation.cc:299] Negotiation successful
0811 06:30:37.357961 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":431,"thread_start_us":295,"threads_started":1}
I20260811 06:30:37.359757 10776 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:30:37.444375 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 10594
I20260811 06:30:37.468209 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 10792
Aug 11 06:30:37 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429834, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.239.195@KRBTEST.COM
I20260811 06:30:37.484539 11003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:37.469278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44816 (local address 127.28.239.195:42883)
0811 06:30:37.469868 (+ 590us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:37.469875 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:37.469907 (+ 32us) server_negotiation.cc:407] Connection header received
0811 06:30:37.469968 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:37.469974 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:37.470225 (+ 251us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:37.470356 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:37.471654 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:37.472633 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:37.473965 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:37.474514 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:37.474621 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:37.479034 (+ 4413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:37.479045 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:37.479061 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:30:37.479094 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:37.482110 (+ 3016us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:37.482724 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:37.482731 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:37.482733 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:37.482849 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:37.483185 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:37.483188 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:37.483190 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:37.483534 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:37.483769 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:37.484018 (+ 249us) server_negotiation.cc:299] Negotiation successful
0811 06:30:37.484241 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":443,"thread_start_us":304,"threads_started":1}
I20260811 06:30:37.485953 10973 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:30:37.571686 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 10792
I20260811 06:30:37.597141 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 10247
I20260811 06:30:37.598380 10338 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:30:37.704928 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 10247
I20260811 06:30:37.736613 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:35823
--webserver_interface=127.28.239.254
--webserver_port=39475
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab
--principal=kudu/127.28.239.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:42763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.239.254:35823
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:30:38.087955 11007 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:30:38.088392 11007 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:30:38.088564 11007 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:30:38.095818 11007 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:30:38.095983 11007 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:30:38.096102 11007 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:30:38.096208 11007 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:30:38.096297 11007 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:30:38.106211 11007 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:42763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.239.254:35823
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab
--principal=kudu/127.28.239.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:35823
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.239.254
--webserver_port=39475
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:30:38.107662 11007 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:30:38.109055 11007 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:30:38.121600 11016 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:30:38.121587 11017 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:30:38.121955 11020 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:30:38.122496 11007 server_base.cc:1034] running on GCE node
I20260811 06:30:38.123315 11007 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:30:38.124893 11007 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:30:38.126102 11007 hybrid_clock.cc:648] HybridClock initialized: now 1786429838126048 us; error 48 us; skew 500 ppm
I20260811 06:30:38.126494 11007 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:30:38 dist-test-slave-gkw7 krb5kdc[10228](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429838, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:30:38.135622 11007 init.cc:377] Logged in from keytab as kudu/127.28.239.254@KRBTEST.COM (short username kudu)
I20260811 06:30:38.137362 11007 webserver.cc:460] Webserver started at http://127.28.239.254:39475/ using document root <none> and password file <none>
I20260811 06:30:38.138163 11007 fs_manager.cc:359] Metadata directory not provided
I20260811 06:30:38.138329 11007 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:30:38.144236 11007 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 06:30:38.147398 11031 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:38.148952 11007 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260811 06:30:38.149144 11007 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/master-0/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/master-0/wal
uuid: "f9b54697de114ef486497b5f41b0d437"
format_stamp: "Formatted at 2026-08-11 06:30:35 on dist-test-slave-gkw7"
server_key: "1905875145f45d68e63839db9086fad7"
server_key_iv: "bac95d0fa3d8f6b7b35ff913f51ea1da"
server_key_version: "encryptionkey@0"
I20260811 06:30:38.150241 11007 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:30:38.164075 11007 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:30:38.168233 11007 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:30:38.168526 11007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:30:38.187651 11007 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.254:35823
I20260811 06:30:38.187677 11102 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.254:35823 every 8 connection(s)
I20260811 06:30:38.189255 11007 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
I20260811 06:30:38.196051 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 11007
I20260811 06:30:38.197216 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.193:43093
--local_ip_for_outbound_sockets=127.28.239.193
--tserver_master_addrs=127.28.239.254:35823
--webserver_port=37851
--webserver_interface=127.28.239.193
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.193.keytab
--principal=kudu/127.28.239.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:42763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.28.239.193
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/krb5kdc/oryx_127.28.239.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 06:30:38.197762 11103 sys_catalog.cc:263] Verifying existing consensus state
I20260811 06:30:38.203722 11103 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9b54697de114ef486497b5f41b0d437: Bootstrap starting.
I20260811 06:30:38.226934 11103 log.cc:824] T 00000000000000000000000000000000 P f9b54697de114ef486497b5f41b0d437: Log is configured to *not* fsync() on all Append() calls
I20260811 06:30:38.237289 11103 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9b54697de114ef486497b5f41b0d437: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 06:30:38.237955 11103 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9b54697de114ef486497b5f41b0d437: Bootstrap complete.
I20260811 06:30:38.248375 11103 raft_consensus.cc:348] T 00000000000000000000000000000000 P f9b54697de114ef486497b5f41b0d437 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9b54697de114ef486497b5f41b0d437" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 35823 } }
I20260811 06:30:38.249322 11103 raft_consensus.cc:729] T 00000000000000000000000000000000 P f9b54697de114ef486497b5f41b0d437 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9b54697de114ef486497b5f41b0d437, State: Initialized, Role: FOLLOWER
I20260811 06:30:38.250203 11103 consensus_queue.cc:260] T 00000000000000000000000000000000 P f9b54697de114ef486497b5f41b0d437 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9b54697de114ef486497b5f41b0d437" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 35823 } }
I20260811 06:30:38.250553 11103 raft_consensus.cc:388] T 00000000000000000000000000000000 P f9b54697de114ef486497b5f41b0d437 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:30:38.250723 11103 raft_consensus.cc:482] T 00000000000000000000000000000000 P f9b54697de114ef486497b5f41b0d437 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:30:38.250945 11103 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f9b54697de114ef486497b5f41b0d437 [term 1 FOLLOWER]: Advancing to term 2
I20260811 06:30:38.255836 11103 raft_consensus.cc:504] T 00000000000000000000000000000000 P f9b54697de114ef486497b5f41b0d437 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9b54697de114ef486497b5f41b0d437" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 35823 } }
I20260811 06:30:38.256444 11103 leader_election.cc:302] T 00000000000000000000000000000000 P f9b54697de114ef486497b5f41b0d437 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f9b54697de114ef486497b5f41b0d437; no voters:
I20260811 06:30:38.257257 11103 leader_election.cc:288] T 00000000000000000000000000000000 P f9b54697de114ef486497b5f41b0d437 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 06:30:38.257421 11107 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f9b54697de114ef486497b5f41b0d437 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 06:30:38.260114 11103 sys_catalog.cc:564] T 00000000000000000000000000000000 P f9b54697de114ef486497b5f41b0d437 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:30:38.260584 11107 raft_consensus.cc:686] T 00000000000000000000000000000000 P f9b54697de114ef486497b5f41b0d437 [term 2 LEADER]: Becoming Leader. State: Replica: f9b54697de114ef486497b5f41b0d437, State: Running, Role: LEADER
I20260811 06:30:38.261364 11107 consensus_queue.cc:237] T 00000000000000000000000000000000 P f9b54697de114ef486497b5f41b0d437 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9b54697de114ef486497b5f41b0d437" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 35823 } }
I20260811 06:30:38.264276 11109 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9b54697de114ef486497b5f41b0d437 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f9b54697de114ef486497b5f41b0d437. Latest consensus state: current_term: 2 leader_uuid: "f9b54697de114ef486497b5f41b0d437" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9b54697de114ef486497b5f41b0d437" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 35823 } } }
I20260811 06:30:38.264619 11109 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9b54697de114ef486497b5f41b0d437 [sys.catalog]: This master's current role is: LEADER
I20260811 06:30:38.265403 11108 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9b54697de114ef486497b5f41b0d437 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "f9b54697de114ef486497b5f41b0d437" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9b54697de114ef486497b5f41b0d437" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 35823 } } }
I20260811 06:30:38.266047 11108 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9b54697de114ef486497b5f41b0d437 [sys.catalog]: This master's current role is: LEADER
I20260811 06:30:38.274780 11115 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:30:38.283102 11115 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:30:38.285308 11115 catalog_manager.cc:1259] Loaded cluster ID: 227ea0dd9739483084e87e24c6b415b2
I20260811 06:30:38.285579 11115 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:30:38.291429 11115 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:30:38.293979 11115 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f9b54697de114ef486497b5f41b0d437: Loaded TSK: 0
I20260811 06:30:38.296252 11115 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 06:30:38.571362 11105 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:30:38.571710 11105 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:30:38.571885 11105 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:30:38.579506 11105 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:30:38.579675 11105 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:30:38.579857 11105 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.193
I20260811 06:30:38.590276 11105 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:42763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/krb5kdc/oryx_127.28.239.193.keytab
--principal=oryx/127.28.239.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.193:43093
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.239.193
--webserver_port=37851
--tserver_master_addrs=127.28.239.254:35823
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.193
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:30:38.591750 11105 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:30:38.593140 11105 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:30:38.609623 11144 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:30:38.609617 11145 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:30:38.610520 11147 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:30:38.611048 11105 server_base.cc:1034] running on GCE node
I20260811 06:30:38.611917 11105 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:30:38.613211 11105 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:30:38.614470 11105 hybrid_clock.cc:648] HybridClock initialized: now 1786429838614396 us; error 61 us; skew 500 ppm
I20260811 06:30:38.614872 11105 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:30:38 dist-test-slave-gkw7 krb5kdc[10228](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429838, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.239.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:30:38.618680 11105 init.cc:377] Logged in from keytab as oryx/127.28.239.193@KRBTEST.COM (short username oryx)
I20260811 06:30:38.620322 11105 webserver.cc:460] Webserver started at http://127.28.239.193:37851/ using document root <none> and password file <none>
I20260811 06:30:38.621060 11105 fs_manager.cc:359] Metadata directory not provided
I20260811 06:30:38.621223 11105 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:30:38.627071 11105 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 06:30:38.630141 11159 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:38.631641 11105 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260811 06:30:38.631834 11105 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/ts-0/wal
uuid: "ad25e7e7049648fb9c3a28fd6cef9d5d"
format_stamp: "Formatted at 2026-08-11 06:30:35 on dist-test-slave-gkw7"
server_key: "a28c363740db6c6f722a790d83699598"
server_key_iv: "3fe06c78ef7663e6df19aaa96e5a073c"
server_key_version: "encryptionkey@0"
I20260811 06:30:38.632822 11105 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:30:38.652347 11105 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:30:38.657202 11105 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:30:38.657495 11105 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:30:38.658689 11105 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:30:38.660197 11105 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:30:38.660329 11105 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:38.660463 11105 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:30:38.660593 11105 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:38.695067 11105 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.193:43093
I20260811 06:30:38.695134 11334 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.193:43093 every 8 connection(s)
I20260811 06:30:38.696615 11105 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
I20260811 06:30:38.699860 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 11105
I20260811 06:30:38.700699 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.194:44689
--local_ip_for_outbound_sockets=127.28.239.194
--tserver_master_addrs=127.28.239.254:35823
--webserver_port=44739
--webserver_interface=127.28.239.194
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.194.keytab
--principal=kudu/127.28.239.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:42763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.28.239.194
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/krb5kdc/oryx_127.28.239.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 06:30:38 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.193@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:30:38 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.193@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:30:38.716467 11346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:38.700050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:35823 (local address 127.28.239.193:57491)
0811 06:30:38.701002 (+ 952us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:38.701056 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:38.702871 (+ 1815us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:38.704339 (+ 1468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:38.704353 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:38.705007 (+ 654us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:38.705625 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:38.705646 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:38.707244 (+ 1598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:38.707250 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:38.708492 (+ 1242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:38.708505 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:38.709601 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:38.709610 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:38.709927 (+ 317us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:38.710742 (+ 815us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:30:38.710779 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:38.715357 (+ 4578us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.239.254:35823: Server oryx/127.28.239.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":613,"thread_start_us":374,"threads_started":1}
W20260811 06:30:38.717207 11339 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:30:38.699791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.193:57491 (local address 127.28.239.254:35823)
0811 06:30:38.700577 (+ 786us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:38.700593 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:38.701209 (+ 616us) server_negotiation.cc:407] Connection header received
0811 06:30:38.703463 (+ 2254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:38.703476 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:38.703819 (+ 343us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:38.704195 (+ 376us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:38.705834 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:38.707069 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:38.708648 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:38.709320 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:38.709584 (+ 264us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:38.715841 (+ 6257us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.239.193:57491: BlockingRecv error: recv got EOF from 127.28.239.193:57491 (error 108)
Metrics: {"server-negotiator.queue_time_us":515,"thread_start_us":309,"threads_started":1}
W20260811 06:30:38.717406 11335 heartbeater.cc:644] Failed to heartbeat to 127.28.239.254:35823 (0 consecutive failures): Not authorized: Failed to ping master at 127.28.239.254:35823: Client connection negotiation failed: client connection to 127.28.239.254:35823: Server oryx/127.28.239.254@KRBTEST.COM not found in Kerberos database
Aug 11 06:30:38 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.193@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:30:38 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.193@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:30:38.728467 11346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:38.718160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:35823 (local address 127.28.239.193:36687)
0811 06:30:38.718384 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:38.718407 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:38.718527 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:38.718877 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:38.718884 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:38.719362 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:38.719577 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:38.719591 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:38.720712 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:38.720721 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:38.722462 (+ 1741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:38.722480 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:38.723261 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:38.723270 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:38.723510 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:38.724221 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:30:38.724250 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:38.728072 (+ 3822us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.239.254:35823: Server oryx/127.28.239.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":68}
W20260811 06:30:38.728497 11339 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:30:38.718352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.193:36687 (local address 127.28.239.254:35823)
0811 06:30:38.718523 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:38.718527 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:38.718545 (+ 18us) server_negotiation.cc:407] Connection header received
0811 06:30:38.718659 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:38.718662 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:38.718720 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:38.718802 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:38.719743 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:38.720553 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:38.722640 (+ 2087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:38.723059 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:38.723201 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:38.728312 (+ 5111us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.239.193:36687: BlockingRecv error: recv got EOF from 127.28.239.193:36687 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 11 06:30:38 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.193@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:30:38 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.193@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:30:38.738364 11346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:38.729480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:35823 (local address 127.28.239.193:34243)
0811 06:30:38.729713 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:38.729735 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:38.729848 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:38.730281 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:38.730284 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:38.730566 (+ 282us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:38.730736 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:38.730745 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:38.732358 (+ 1613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:38.732362 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:38.733279 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:38.733290 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:38.734279 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:38.734283 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:38.734448 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:38.735100 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:30:38.735117 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:38.738127 (+ 3010us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.239.254:35823: Server oryx/127.28.239.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":86}
W20260811 06:30:38.738623 11339 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:30:38.729690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.193:34243 (local address 127.28.239.254:35823)
0811 06:30:38.729906 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:38.729914 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:38.729936 (+ 22us) server_negotiation.cc:407] Connection header received
0811 06:30:38.730000 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:38.730007 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:38.730129 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:38.730250 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:38.730885 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:38.732190 (+ 1305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:38.733425 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:38.734120 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:38.734241 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:38.738352 (+ 4111us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.239.193:34243: BlockingRecv error: recv got EOF from 127.28.239.193:34243 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260811 06:30:38.739153 11335 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 06:30:39.063576 11347 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:30:39.063907 11347 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:30:39.064088 11347 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:30:39.071575 11347 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:30:39.071694 11347 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:30:39.071888 11347 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.194
I20260811 06:30:39.082141 11347 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:42763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/krb5kdc/oryx_127.28.239.194.keytab
--principal=oryx/127.28.239.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.194:44689
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.239.194
--webserver_port=44739
--tserver_master_addrs=127.28.239.254:35823
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.194
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:30:39.083591 11347 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:30:39.084956 11347 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:30:39.098882 11353 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:30:39.098992 11356 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:30:39.098966 11354 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:30:39.100003 11347 server_base.cc:1034] running on GCE node
I20260811 06:30:39.100626 11347 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:30:39.101725 11347 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:30:39.102990 11347 hybrid_clock.cc:648] HybridClock initialized: now 1786429839102936 us; error 47 us; skew 500 ppm
I20260811 06:30:39.103348 11347 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:30:39 dist-test-slave-gkw7 krb5kdc[10228](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429839, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.239.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:30:39.106889 11347 init.cc:377] Logged in from keytab as oryx/127.28.239.194@KRBTEST.COM (short username oryx)
I20260811 06:30:39.108500 11347 webserver.cc:460] Webserver started at http://127.28.239.194:44739/ using document root <none> and password file <none>
I20260811 06:30:39.109243 11347 fs_manager.cc:359] Metadata directory not provided
I20260811 06:30:39.109405 11347 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:30:39.115109 11347 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 06:30:39.118115 11365 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:39.119710 11347 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260811 06:30:39.119910 11347 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/ts-1/wal
uuid: "f283822b11c34681844bde87d93c30d1"
format_stamp: "Formatted at 2026-08-11 06:30:36 on dist-test-slave-gkw7"
server_key: "959ba17aebe2902b3777ab49d034c1b8"
server_key_iv: "ec9920d68de8f24cd4ed02aa0028e28c"
server_key_version: "encryptionkey@0"
I20260811 06:30:39.120942 11347 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:30:39.133528 11347 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:30:39.138044 11347 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:30:39.138345 11347 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:30:39.139393 11347 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:30:39.140836 11347 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:30:39.140930 11347 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:39.141058 11347 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:30:39.141197 11347 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:39.176857 11347 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.194:44689
I20260811 06:30:39.176961 11538 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.194:44689 every 8 connection(s)
I20260811 06:30:39.178500 11347 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
I20260811 06:30:39.184775 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 11347
I20260811 06:30:39.185611 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.195:42883
--local_ip_for_outbound_sockets=127.28.239.195
--tserver_master_addrs=127.28.239.254:35823
--webserver_port=44917
--webserver_interface=127.28.239.195
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.195.keytab
--principal=kudu/127.28.239.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:42763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.28.239.195
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/krb5kdc/oryx_127.28.239.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 06:30:39 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.194@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:30:39 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.194@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
W20260811 06:30:39.198289 11339 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:30:39.181247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.194:44333 (local address 127.28.239.254:35823)
0811 06:30:39.181434 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:39.181439 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:39.182678 (+ 1239us) server_negotiation.cc:407] Connection header received
0811 06:30:39.184588 (+ 1910us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:39.184598 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:39.184714 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:39.184860 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:39.186778 (+ 1918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:39.188171 (+ 1393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:39.190649 (+ 2478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:39.191524 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:39.191694 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:39.198022 (+ 6328us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.239.194:44333: BlockingRecv error: recv got EOF from 127.28.239.194:44333 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 06:30:39.200176 11542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:39.181736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:35823 (local address 127.28.239.194:44333)
0811 06:30:39.182483 (+ 747us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:39.182525 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:39.184257 (+ 1732us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:39.185185 (+ 928us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:39.185194 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:39.185956 (+ 762us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:39.186566 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:39.186583 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:39.188378 (+ 1795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:39.188384 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:39.190457 (+ 2073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:39.190476 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:39.191815 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:39.191822 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:39.192116 (+ 294us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:39.193011 (+ 895us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:30:39.193042 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:39.198837 (+ 5795us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.239.254:35823: Server oryx/127.28.239.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":289,"threads_started":1}
W20260811 06:30:39.201198 11540 heartbeater.cc:644] Failed to heartbeat to 127.28.239.254:35823 (0 consecutive failures): Not authorized: Failed to ping master at 127.28.239.254:35823: Client connection negotiation failed: client connection to 127.28.239.254:35823: Server oryx/127.28.239.254@KRBTEST.COM not found in Kerberos database
Aug 11 06:30:39 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.194@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:30:39 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.194@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:30:39.213145 11542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:39.201969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:35823 (local address 127.28.239.194:37951)
0811 06:30:39.202309 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:39.202332 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:39.202499 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:39.202965 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:39.202974 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:39.203550 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:39.203807 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:39.203825 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:39.204996 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:39.205005 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:39.206926 (+ 1921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:39.206947 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:39.207731 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:39.207740 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:39.207993 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:39.208724 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:30:39.208755 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:39.212790 (+ 4035us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.239.254:35823: Server oryx/127.28.239.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":179}
W20260811 06:30:39.213364 11339 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:30:39.202129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.194:37951 (local address 127.28.239.254:35823)
0811 06:30:39.202287 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:39.202291 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:39.202495 (+ 204us) server_negotiation.cc:407] Connection header received
0811 06:30:39.202656 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:39.202666 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:39.202763 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:39.202882 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:39.204001 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:39.204824 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:39.207118 (+ 2294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:39.207547 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:39.207656 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:39.213052 (+ 5396us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.239.194:37951: BlockingRecv error: recv got EOF from 127.28.239.194:37951 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 11 06:30:39 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.194@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:30:39 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.194@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:30:39.223145 11542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:39.214392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:35823 (local address 127.28.239.194:57973)
0811 06:30:39.214609 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:39.214632 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:39.214756 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:39.215144 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:39.215149 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:39.215497 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:39.215664 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:39.215673 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:39.216815 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:39.216823 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:39.217940 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:39.217953 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:39.218727 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:39.218734 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:39.218936 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:39.219609 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:30:39.219630 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:39.222800 (+ 3170us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.239.254:35823: Server oryx/127.28.239.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":70}
W20260811 06:30:39.223232 11339 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:30:39.214519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.194:57973 (local address 127.28.239.254:35823)
0811 06:30:39.214740 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:39.214746 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:39.214767 (+ 21us) server_negotiation.cc:407] Connection header received
0811 06:30:39.214908 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:39.214914 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:39.214987 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:39.215104 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:39.215828 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:39.216654 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:39.218096 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:39.218549 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:39.218714 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:39.223009 (+ 4295us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.239.194:57973: BlockingRecv error: recv got EOF from 127.28.239.194:57973 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
W20260811 06:30:39.223771 11540 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 06:30:39.552487 11543 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:30:39.552825 11543 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:30:39.552991 11543 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:30:39.560911 11543 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:30:39.561070 11543 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:30:39.561497 11543 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.195
I20260811 06:30:39.572330 11543 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:42763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/krb5kdc/oryx_127.28.239.195.keytab
--principal=oryx/127.28.239.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.195:42883
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.239.195
--webserver_port=44917
--tserver_master_addrs=127.28.239.254:35823
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.195
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:30:39.573904 11543 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:30:39.575342 11543 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:30:39.589879 11548 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:30:39.589808 11549 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:30:39.589789 11551 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:30:39.591159 11543 server_base.cc:1034] running on GCE node
I20260811 06:30:39.591750 11543 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:30:39.592717 11543 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:30:39.593942 11543 hybrid_clock.cc:648] HybridClock initialized: now 1786429839593894 us; error 45 us; skew 500 ppm
I20260811 06:30:39.594304 11543 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:30:39 dist-test-slave-gkw7 krb5kdc[10228](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429839, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.239.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:30:39.598189 11543 init.cc:377] Logged in from keytab as oryx/127.28.239.195@KRBTEST.COM (short username oryx)
I20260811 06:30:39.599893 11543 webserver.cc:460] Webserver started at http://127.28.239.195:44917/ using document root <none> and password file <none>
I20260811 06:30:39.600642 11543 fs_manager.cc:359] Metadata directory not provided
I20260811 06:30:39.600805 11543 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:30:39.606618 11543 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 06:30:39.609480 11560 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:39.611186 11543 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 06:30:39.611379 11543 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/ts-2/wal
uuid: "04c5ac8768c84023a5457f0b2617ef25"
format_stamp: "Formatted at 2026-08-11 06:30:36 on dist-test-slave-gkw7"
server_key: "35e3694c2c4ccb73b7589e2a08544128"
server_key_iv: "c5d89d381af8fd7abd67dca2cc74b769"
server_key_version: "encryptionkey@0"
I20260811 06:30:39.612403 11543 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:30:39.653326 11543 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:30:39.660100 11543 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:30:39.660475 11543 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:30:39.661906 11543 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:30:39.663755 11543 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:30:39.663890 11543 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:39.664021 11543 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:30:39.664110 11543 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:39.712708 11543 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.195:42883
I20260811 06:30:39.712814 11729 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.195:42883 every 8 connection(s)
I20260811 06:30:39.714315 11543 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
I20260811 06:30:39.721650 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 11543
Aug 11 06:30:39 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.195@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:30:39 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.195@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
W20260811 06:30:39.731779 11339 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:30:39.717052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.195:37205 (local address 127.28.239.254:35823)
0811 06:30:39.717240 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:39.717255 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:39.718498 (+ 1243us) server_negotiation.cc:407] Connection header received
0811 06:30:39.720371 (+ 1873us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:39.720377 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:39.720482 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:39.720597 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:39.722631 (+ 2034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:39.723995 (+ 1364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:39.725753 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:39.726212 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:39.726379 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:39.731447 (+ 5068us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.239.195:37205: BlockingRecv error: recv got EOF from 127.28.239.195:37205 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 06:30:39.732550 11734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:39.717519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:35823 (local address 127.28.239.195:37205)
0811 06:30:39.718330 (+ 811us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:39.718375 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:39.720085 (+ 1710us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:39.720848 (+ 763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:39.720857 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:39.721648 (+ 791us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:39.722453 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:39.722474 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:39.724159 (+ 1685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:39.724164 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:39.725619 (+ 1455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:39.725632 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:39.726471 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:39.726479 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:39.726711 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:39.727410 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:30:39.727437 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:39.731242 (+ 3805us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.239.254:35823: Server oryx/127.28.239.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":295,"threads_started":1}
W20260811 06:30:39.733273 11731 heartbeater.cc:644] Failed to heartbeat to 127.28.239.254:35823 (0 consecutive failures): Not authorized: Failed to ping master at 127.28.239.254:35823: Client connection negotiation failed: client connection to 127.28.239.254:35823: Server oryx/127.28.239.254@KRBTEST.COM not found in Kerberos database
I20260811 06:30:39.738699 11737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:39.722830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51570 (local address 127.28.239.254:35823)
0811 06:30:39.724019 (+ 1189us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:39.724023 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:39.724039 (+ 16us) server_negotiation.cc:407] Connection header received
0811 06:30:39.724125 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:39.724130 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:39.724210 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:39.724344 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:39.726121 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:39.726933 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:39.728233 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:39.728656 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:39.728787 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:39.731576 (+ 2789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:39.731587 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:39.731604 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:30:39.731653 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:39.735180 (+ 3527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:39.735863 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:39.735873 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:39.735888 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:39.736112 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:30:39.736538 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:30:39.736542 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:30:39.736544 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:30:39.737033 (+ 489us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:39.737356 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:39.737893 (+ 537us) server_negotiation.cc:299] Negotiation successful
0811 06:30:39.738286 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1045,"thread_start_us":897,"threads_started":1}
Aug 11 06:30:39 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.195@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:30:39 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.195@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:30:39.742522 11734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:39.733957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:35823 (local address 127.28.239.195:57183)
0811 06:30:39.734153 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:39.734172 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:39.734295 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:39.734634 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:39.734639 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:39.734972 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:39.735102 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:39.735110 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:39.736158 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:39.736165 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:39.737392 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:39.737406 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:39.738391 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:39.738396 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:39.738522 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:39.739048 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:30:39.739069 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:39.742230 (+ 3161us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.239.254:35823: Server oryx/127.28.239.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":60}
W20260811 06:30:39.743875 11339 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:30:39.734063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.195:57183 (local address 127.28.239.254:35823)
0811 06:30:39.734235 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:39.734240 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:39.734257 (+ 17us) server_negotiation.cc:407] Connection header received
0811 06:30:39.734441 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:39.734447 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:39.734511 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:39.734610 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:39.735238 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:39.735952 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:39.737518 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:39.738251 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:39.738381 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:39.742407 (+ 4026us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.239.195:57183: BlockingRecv error: recv got EOF from 127.28.239.195:57183 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 11 06:30:39 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.193@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:30:39 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.193@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:30:39 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.195@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:30:39.752149 11738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:39.739792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:35823 (local address 127.28.239.193:47103)
0811 06:30:39.741979 (+ 2187us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:39.741995 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:39.742206 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:39.742792 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:39.742796 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:39.743387 (+ 591us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:39.743706 (+ 319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:39.743719 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:39.744929 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:39.744934 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:39.746369 (+ 1435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:39.746384 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:39.747510 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:39.747515 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:39.747639 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:39.748256 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:30:39.748280 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:39.751817 (+ 3537us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.239.254:35823: Server oryx/127.28.239.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":2058,"thread_start_us":1927,"threads_started":1}
Aug 11 06:30:39 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.195@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
W20260811 06:30:39.752331 11737 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:30:39.739800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.193:47103 (local address 127.28.239.254:35823)
0811 06:30:39.740077 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:39.740085 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:39.742086 (+ 2001us) server_negotiation.cc:407] Connection header received
0811 06:30:39.742397 (+ 311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:39.742403 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:39.742475 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:39.742545 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:39.743873 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:39.744786 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:39.746540 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:39.747319 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:39.747489 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:39.752086 (+ 4597us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.239.193:47103: BlockingRecv error: recv got EOF from 127.28.239.193:47103 (error 108)
Metrics: {"server-negotiator.queue_time_us":138}
I20260811 06:30:39.753396 11734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:39.743150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:35823 (local address 127.28.239.195:54089)
0811 06:30:39.743383 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:39.743401 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:39.743493 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:39.744427 (+ 934us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:39.744435 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:39.744936 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:39.745206 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:39.745222 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:39.746696 (+ 1474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:39.746702 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:39.748114 (+ 1412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:39.748128 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:39.749047 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:39.749052 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:39.749222 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:39.749861 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:30:39.749878 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:39.753179 (+ 3301us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.239.254:35823: Server oryx/127.28.239.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131}
W20260811 06:30:39.753717 11339 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:30:39.743244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.195:54089 (local address 127.28.239.254:35823)
0811 06:30:39.744134 (+ 890us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:39.744140 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:39.744158 (+ 18us) server_negotiation.cc:407] Connection header received
0811 06:30:39.744200 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:39.744205 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:39.744264 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:39.744359 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:39.745380 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:39.746547 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:39.748265 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:39.748890 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:39.749015 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:39.753344 (+ 4329us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.239.195:54089: BlockingRecv error: recv got EOF from 127.28.239.195:54089 (error 108)
Metrics: {"server-negotiator.queue_time_us":805,"thread_start_us":2503,"threads_started":1}
W20260811 06:30:39.753816 11731 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 11 06:30:40 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.194@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:30:40 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.194@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:30:40.234494 11740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:40.224512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:35823 (local address 127.28.239.194:52075)
0811 06:30:40.225136 (+ 624us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:40.225154 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:40.225336 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:40.225829 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:40.225834 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:40.226396 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:40.226633 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:40.226644 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:40.227790 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:40.227796 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:40.228896 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:40.228909 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:40.229822 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:40.229830 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:40.230021 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:40.230643 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:30:40.230663 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:40.234261 (+ 3598us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.239.254:35823: Server oryx/127.28.239.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":319,"threads_started":1}
W20260811 06:30:40.234797 11339 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:30:40.224597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.194:52075 (local address 127.28.239.254:35823)
0811 06:30:40.224891 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:40.224900 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:40.225256 (+ 356us) server_negotiation.cc:407] Connection header received
0811 06:30:40.225489 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:40.225495 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:40.225604 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:40.225733 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:40.226800 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:40.227625 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:40.229070 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:40.229613 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:40.229755 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:40.234508 (+ 4753us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.239.194:52075: BlockingRecv error: recv got EOF from 127.28.239.194:52075 (error 108)
Metrics: {"server-negotiator.queue_time_us":115}
Aug 11 06:30:40 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.193@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:30:40 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.195@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:30:40 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.193@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:30:40 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.195@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:30:40.764845 11742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:40.753354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:35823 (local address 127.28.239.193:58351)
0811 06:30:40.754019 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:40.754043 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:40.754256 (+ 213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:40.754705 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:40.754713 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:40.755486 (+ 773us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:40.755667 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:40.755676 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:40.756968 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:40.756973 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:40.758324 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:40.758341 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:40.759142 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:40.759149 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:40.759358 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:40.760158 (+ 800us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:30:40.760178 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:40.764468 (+ 4290us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.239.254:35823: Server oryx/127.28.239.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":320,"threads_started":1}
W20260811 06:30:40.764969 11744 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:30:40.753475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.193:58351 (local address 127.28.239.254:35823)
0811 06:30:40.754084 (+ 609us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:40.754089 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:40.754205 (+ 116us) server_negotiation.cc:407] Connection header received
0811 06:30:40.754422 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:40.754428 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:40.754519 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:40.754635 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:40.755803 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:40.756787 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:40.758496 (+ 1709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:40.758951 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:40.759052 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:40.764698 (+ 5646us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.239.193:58351: BlockingRecv error: recv got EOF from 127.28.239.193:58351 (error 108)
Metrics: {"server-negotiator.queue_time_us":460,"thread_start_us":283,"threads_started":1}
I20260811 06:30:40.766183 11750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:40.754593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:35823 (local address 127.28.239.195:57249)
0811 06:30:40.755332 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:40.755357 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:40.755495 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:40.755987 (+ 492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:40.755991 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:40.756448 (+ 457us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:40.756651 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:40.756660 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:40.758104 (+ 1444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:40.758112 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:40.759454 (+ 1342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:40.759474 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:40.760599 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:40.760610 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:40.760831 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:40.761594 (+ 763us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:30:40.761620 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:40.765814 (+ 4194us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.239.254:35823: Server oryx/127.28.239.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":584,"thread_start_us":433,"threads_started":1}
W20260811 06:30:40.766390 11747 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:30:40.754615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.195:57249 (local address 127.28.239.254:35823)
0811 06:30:40.755644 (+ 1029us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:40.755652 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:40.755677 (+ 25us) server_negotiation.cc:407] Connection header received
0811 06:30:40.755715 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:40.755718 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:40.755808 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:40.755934 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:40.756829 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:40.757939 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:40.759666 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:40.760303 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:40.760500 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:40.766064 (+ 5564us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.239.195:57249: BlockingRecv error: recv got EOF from 127.28.239.195:57249 (error 108)
Metrics: {"server-negotiator.queue_time_us":882,"thread_start_us":721,"threads_started":1}
Aug 11 06:30:41 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.194@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:30:41 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.194@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:30:41.244385 11751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:41.235567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:35823 (local address 127.28.239.194:52067)
0811 06:30:41.236180 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:41.236200 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:41.236369 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:41.236704 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:41.236708 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:41.237123 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:41.237357 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:41.237369 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:41.238436 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:41.238440 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:41.239336 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:41.239347 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:41.239989 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:41.239993 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:41.240105 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:41.240768 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:30:41.240787 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:41.244132 (+ 3345us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.239.254:35823: Server oryx/127.28.239.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":357,"threads_started":1}
W20260811 06:30:41.244586 11747 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:30:41.235667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.194:52067 (local address 127.28.239.254:35823)
0811 06:30:41.235857 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:41.235863 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:41.236378 (+ 515us) server_negotiation.cc:407] Connection header received
0811 06:30:41.236499 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:41.236503 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:41.236568 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:41.236659 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:41.237562 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:41.238317 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:41.239464 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:41.239858 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:41.239950 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:41.244344 (+ 4394us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.239.194:52067: BlockingRecv error: recv got EOF from 127.28.239.194:52067 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 11 06:30:41 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.193@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:30:41 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.193@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:30:41 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.195@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:30:41.776304 11752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:41.765837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:35823 (local address 127.28.239.193:40923)
0811 06:30:41.766360 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:41.766377 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:41.766485 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:41.767022 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:41.767029 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:41.767588 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:41.767872 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:41.767885 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:41.769400 (+ 1515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:41.769405 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:41.770868 (+ 1463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:41.770881 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:41.771587 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:41.771593 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:41.771795 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:41.772469 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:30:41.772488 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:41.775928 (+ 3440us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.239.254:35823: Server oryx/127.28.239.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":267,"threads_started":1}
W20260811 06:30:41.776459 11753 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:30:41.765947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.193:40923 (local address 127.28.239.254:35823)
0811 06:30:41.766711 (+ 764us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:41.766715 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:41.766731 (+ 16us) server_negotiation.cc:407] Connection header received
0811 06:30:41.766775 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:41.766778 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:41.766847 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:41.766941 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:41.768050 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:41.769234 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:41.771006 (+ 1772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:41.771433 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:41.771571 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:41.776134 (+ 4563us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.239.193:40923: BlockingRecv error: recv got EOF from 127.28.239.193:40923 (error 108)
Metrics: {"server-negotiator.queue_time_us":646,"thread_start_us":531,"threads_started":1}
Aug 11 06:30:41 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.195@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:30:41.778311 11754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:41.767193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:35823 (local address 127.28.239.195:46791)
0811 06:30:41.767953 (+ 760us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:41.767988 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:41.768144 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:41.768490 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:41.768495 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:41.769013 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:41.769222 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:41.769234 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:41.770781 (+ 1547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:41.770791 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:41.772019 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:41.772034 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:41.772888 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:41.772896 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:41.773078 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:41.774159 (+ 1081us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:30:41.774176 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:41.778060 (+ 3884us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.239.254:35823: Server oryx/127.28.239.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":663,"thread_start_us":537,"threads_started":1}
W20260811 06:30:41.778577 11755 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:30:41.767256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.195:46791 (local address 127.28.239.254:35823)
0811 06:30:41.767799 (+ 543us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:41.767806 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:41.768094 (+ 288us) server_negotiation.cc:407] Connection header received
0811 06:30:41.768272 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:41.768275 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:41.768333 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:41.768415 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:41.769401 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:41.770578 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:41.772256 (+ 1678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:41.772712 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:41.772856 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:41.778296 (+ 5440us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.239.195:46791: BlockingRecv error: recv got EOF from 127.28.239.195:46791 (error 108)
Metrics: {"server-negotiator.queue_time_us":411,"thread_start_us":260,"threads_started":1}
Aug 11 06:30:42 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.194@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:30:42 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.194@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:30:42.254725 11757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:42.245465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:35823 (local address 127.28.239.194:60053)
0811 06:30:42.246062 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:42.246079 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:42.246245 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:42.246598 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:42.246603 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:42.246990 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:42.247164 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:42.247173 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:42.248237 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:42.248244 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:42.249171 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:42.249182 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:42.249964 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:42.249970 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:42.250161 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:42.250820 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:30:42.250839 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:42.254452 (+ 3613us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.239.254:35823: Server oryx/127.28.239.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":353,"threads_started":1}
W20260811 06:30:42.254978 11755 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:30:42.245597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.194:60053 (local address 127.28.239.254:35823)
0811 06:30:42.245804 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:42.245813 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:42.246176 (+ 363us) server_negotiation.cc:407] Connection header received
0811 06:30:42.246380 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:42.246386 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:42.246454 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:42.246562 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:42.247332 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:42.248096 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:42.249318 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:42.249804 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:42.249939 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:42.254669 (+ 4730us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.239.194:60053: BlockingRecv error: recv got EOF from 127.28.239.194:60053 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 11 06:30:42 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.193@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:30:42 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.193@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:30:42 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.195@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:30:42.789201 11758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:42.777599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:35823 (local address 127.28.239.193:34297)
0811 06:30:42.778155 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:42.778179 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:42.778305 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:42.778652 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:42.778678 (+ 26us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:42.779168 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:42.779387 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:42.779399 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:42.781674 (+ 2275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:42.781681 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:42.783098 (+ 1417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:42.783113 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:42.784152 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:42.784159 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:42.784368 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:42.785058 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:30:42.785093 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:42.788757 (+ 3664us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.239.254:35823: Server oryx/127.28.239.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":307,"threads_started":1}
W20260811 06:30:42.789304 11759 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:30:42.777676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.193:34297 (local address 127.28.239.254:35823)
0811 06:30:42.778179 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:42.778183 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:42.778273 (+ 90us) server_negotiation.cc:407] Connection header received
0811 06:30:42.778410 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:42.778414 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:42.778479 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:42.778574 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:42.779613 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:42.781501 (+ 1888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:42.783248 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:42.783953 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:42.784061 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:42.789054 (+ 4993us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.239.193:34297: BlockingRecv error: recv got EOF from 127.28.239.193:34297 (error 108)
Metrics: {"mutex_wait_us":7,"server-negotiator.queue_time_us":411,"thread_start_us":282,"threads_started":1}
Aug 11 06:30:42 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.195@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:30:42.790946 11761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:42.779296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:35823 (local address 127.28.239.195:46213)
0811 06:30:42.780743 (+ 1447us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:42.780766 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:42.780961 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:42.781394 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:42.781400 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:42.782079 (+ 679us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:42.782311 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:42.782335 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:42.783853 (+ 1518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:42.783858 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:42.785197 (+ 1339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:42.785215 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:42.786140 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:42.786149 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:42.786362 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:42.787022 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:30:42.787046 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:42.790644 (+ 3598us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.239.254:35823: Server oryx/127.28.239.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1309,"thread_start_us":614,"threads_started":1}
W20260811 06:30:42.791117 11762 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:30:42.779389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.195:46213 (local address 127.28.239.254:35823)
0811 06:30:42.780524 (+ 1135us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:42.780528 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:42.780837 (+ 309us) server_negotiation.cc:407] Connection header received
0811 06:30:42.781107 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:42.781114 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:42.781210 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:42.781346 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:42.782493 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:42.783678 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:42.785374 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:42.785968 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:42.786085 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:42.790844 (+ 4759us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.239.195:46213: BlockingRecv error: recv got EOF from 127.28.239.195:46213 (error 108)
Metrics: {"server-negotiator.queue_time_us":1025,"thread_start_us":685,"threads_started":1}
Aug 11 06:30:43 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.194@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:30:43 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.194@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:30:43.268944 11763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:43.255814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:35823 (local address 127.28.239.194:50557)
0811 06:30:43.256343 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:43.256359 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:43.256526 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:43.256913 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:43.256921 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:43.257331 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:43.257493 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:43.257502 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:43.258585 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:43.258595 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:43.259572 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:43.259584 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:43.260282 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:43.260288 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:43.260483 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:43.265694 (+ 5211us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:30:43.265713 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:43.268689 (+ 2976us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.239.254:35823: Server oryx/127.28.239.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":294,"threads_started":1}
W20260811 06:30:43.269208 11762 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:30:43.255844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.194:50557 (local address 127.28.239.254:35823)
0811 06:30:43.256075 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:43.256084 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:43.256458 (+ 374us) server_negotiation.cc:407] Connection header received
0811 06:30:43.256668 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:43.256672 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:43.256736 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:43.256857 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:43.257666 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:43.258442 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:43.259710 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:43.260130 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:43.260260 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:43.268936 (+ 8676us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.239.194:50557: BlockingRecv error: recv got EOF from 127.28.239.194:50557 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 11 06:30:43 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.193@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:30:43 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.193@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:30:43 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.195@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:30:43.802373 11765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:43.790196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:35823 (local address 127.28.239.193:45863)
0811 06:30:43.790777 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:43.790796 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:43.790906 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:43.791391 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:43.791395 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:43.792738 (+ 1343us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:43.793002 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:43.793019 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:43.794592 (+ 1573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:43.794600 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:43.796253 (+ 1653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:43.796267 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:43.797212 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:43.797217 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:43.797449 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:43.798162 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:30:43.798191 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:43.801995 (+ 3804us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.239.254:35823: Server oryx/127.28.239.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":266,"threads_started":1}
Aug 11 06:30:43 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.195@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
W20260811 06:30:43.802474 11766 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:30:43.790277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.193:45863 (local address 127.28.239.254:35823)
0811 06:30:43.791075 (+ 798us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:43.791081 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:43.791101 (+ 20us) server_negotiation.cc:407] Connection header received
0811 06:30:43.791149 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:43.791153 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:43.791212 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:43.791308 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:43.793170 (+ 1862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:43.794423 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:43.796389 (+ 1966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:43.797045 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:43.797173 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:43.802205 (+ 5032us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.239.193:45863: BlockingRecv error: recv got EOF from 127.28.239.193:45863 (error 108)
Metrics: {"server-negotiator.queue_time_us":696,"thread_start_us":564,"threads_started":1}
I20260811 06:30:43.803527 11770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:43.792031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:35823 (local address 127.28.239.195:46285)
0811 06:30:43.792975 (+ 944us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:43.792996 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:43.793215 (+ 219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:43.793813 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:43.793817 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:43.794273 (+ 456us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:43.794457 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:43.794472 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:43.796090 (+ 1618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:43.796095 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:43.797512 (+ 1417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:43.797524 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:43.798907 (+ 1383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:43.798913 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:43.799162 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:43.799873 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:30:43.799904 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:43.803192 (+ 3288us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.239.254:35823: Server oryx/127.28.239.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":819,"thread_start_us":660,"threads_started":1}
W20260811 06:30:43.803682 11767 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:30:43.792064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.195:46285 (local address 127.28.239.254:35823)
0811 06:30:43.792592 (+ 528us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:43.792598 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:43.793083 (+ 485us) server_negotiation.cc:407] Connection header received
0811 06:30:43.793368 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:43.793372 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:43.793456 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:43.793562 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:43.794630 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:43.795942 (+ 1312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:43.797684 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:43.798482 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:43.798637 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:43.803396 (+ 4759us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.239.195:46285: BlockingRecv error: recv got EOF from 127.28.239.195:46285 (error 108)
Metrics: {"server-negotiator.queue_time_us":429,"thread_start_us":287,"threads_started":1}
Aug 11 06:30:44 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.194@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:30:44 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.194@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:30:44.278767 11771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:44.270097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:35823 (local address 127.28.239.194:39775)
0811 06:30:44.270710 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:44.270726 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:44.270889 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:44.271229 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:44.271234 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:44.271664 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:44.271828 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:44.271838 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:44.272961 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:44.272967 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:44.273928 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:44.273940 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:44.274656 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:44.274662 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:44.274838 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:44.275447 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:30:44.275465 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:44.278431 (+ 2966us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.239.254:35823: Server oryx/127.28.239.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":294,"threads_started":1}
W20260811 06:30:44.279007 11767 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:30:44.270291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.194:39775 (local address 127.28.239.254:35823)
0811 06:30:44.270533 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:44.270539 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:44.270825 (+ 286us) server_negotiation.cc:407] Connection header received
0811 06:30:44.271020 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:44.271026 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:44.271096 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:44.271193 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:44.271977 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:44.272825 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:44.274071 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:44.274510 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:44.274640 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:44.278742 (+ 4102us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.239.194:39775: BlockingRecv error: recv got EOF from 127.28.239.194:39775 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 11 06:30:44 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.193@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:30:44 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.195@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:30:44 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.193@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:30:44 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.195@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:30:44.814009 11775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:44.803446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:35823 (local address 127.28.239.193:47811)
0811 06:30:44.804613 (+ 1167us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:44.804633 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:44.804780 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:44.805425 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:44.805429 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:44.805884 (+ 455us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:44.806040 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:44.806048 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:44.807338 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:44.807344 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:44.808365 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:44.808378 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:44.809449 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:44.809456 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:44.809668 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:44.810423 (+ 755us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:30:44.810441 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:44.813657 (+ 3216us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.239.254:35823: Server oryx/127.28.239.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1039,"thread_start_us":900,"threads_started":1}
W20260811 06:30:44.814167 11776 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:30:44.803485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.193:47811 (local address 127.28.239.254:35823)
0811 06:30:44.804213 (+ 728us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:44.804218 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:44.804920 (+ 702us) server_negotiation.cc:407] Connection header received
0811 06:30:44.804992 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:44.805000 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:44.805111 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:44.805227 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:44.806180 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:44.807183 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:44.808533 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:44.809288 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:44.809392 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:44.813895 (+ 4503us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.239.193:47811: BlockingRecv error: recv got EOF from 127.28.239.193:47811 (error 108)
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":235,"threads_started":1}
W20260811 06:30:44.815025 11778 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:30:44.804605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.195:54771 (local address 127.28.239.254:35823)
0811 06:30:44.805384 (+ 779us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:44.805389 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:44.805402 (+ 13us) server_negotiation.cc:407] Connection header received
0811 06:30:44.805438 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:44.805441 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:44.805499 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:44.805649 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:44.806580 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:44.807501 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:44.809181 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:44.809852 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:44.809992 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:44.814703 (+ 4711us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.239.195:54771: BlockingRecv error: recv got EOF from 127.28.239.195:54771 (error 108)
Metrics: {"server-negotiator.queue_time_us":640,"thread_start_us":494,"threads_started":1}
I20260811 06:30:44.815289 11777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:44.804529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:35823 (local address 127.28.239.195:54771)
0811 06:30:44.805061 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:44.805082 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:44.805226 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:44.805679 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:44.805684 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:44.806178 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:44.806403 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:44.806416 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:44.807676 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:44.807681 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:44.809018 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:44.809035 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:44.810037 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:44.810045 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:44.810273 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:44.810897 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:30:44.810917 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:44.814414 (+ 3497us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.239.254:35823: Server oryx/127.28.239.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":275,"threads_started":1}
Aug 11 06:30:45 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.194@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:30:45 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.194@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:30:45.291342 11779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:45.279875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:35823 (local address 127.28.239.194:42889)
0811 06:30:45.280467 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:45.280489 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:45.280616 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:45.282531 (+ 1915us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:45.282536 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:45.283090 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:45.283291 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:45.283303 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:45.284977 (+ 1674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:45.284987 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:45.286217 (+ 1230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:45.286231 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:45.287055 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:45.287062 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:45.287266 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:45.287881 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:30:45.287900 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:45.291003 (+ 3103us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.239.254:35823: Server oryx/127.28.239.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":310,"threads_started":1}
W20260811 06:30:45.291502 11778 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:30:45.281909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.194:42889 (local address 127.28.239.254:35823)
0811 06:30:45.282122 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:45.282128 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:45.282148 (+ 20us) server_negotiation.cc:407] Connection header received
0811 06:30:45.282215 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:45.282221 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:45.282333 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:45.282471 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:45.283506 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:45.284695 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:45.286405 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:45.286880 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:45.287034 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:45.291229 (+ 4195us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.239.194:42889: BlockingRecv error: recv got EOF from 127.28.239.194:42889 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 11 06:30:45 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.193@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:30:45 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.195@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:30:45 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.193@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:30:45 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.195@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:30:45.827037 11781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:45.815315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:35823 (local address 127.28.239.193:39253)
0811 06:30:45.816274 (+ 959us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:45.816296 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:45.816450 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:45.817022 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:45.817028 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:45.817632 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:45.817900 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:45.817914 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:45.819377 (+ 1463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:45.819382 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:45.820688 (+ 1306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:45.820700 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:45.821811 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:45.821819 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:45.821980 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:45.822707 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:30:45.822733 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:45.826655 (+ 3922us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.239.254:35823: Server oryx/127.28.239.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":803,"thread_start_us":586,"threads_started":1}
W20260811 06:30:45.827178 11780 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:30:45.815366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.193:39253 (local address 127.28.239.254:35823)
0811 06:30:45.816010 (+ 644us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:45.816018 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:45.816379 (+ 361us) server_negotiation.cc:407] Connection header received
0811 06:30:45.816570 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:45.816577 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:45.816670 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:45.816797 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:45.818099 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:45.819233 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:45.820917 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:45.821625 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:45.821749 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:45.826852 (+ 5103us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.239.193:39253: BlockingRecv error: recv got EOF from 127.28.239.193:39253 (error 108)
Metrics: {"server-negotiator.queue_time_us":505,"thread_start_us":370,"threads_started":1}
I20260811 06:30:45.827353 11782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:45.816297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:35823 (local address 127.28.239.195:43069)
0811 06:30:45.816920 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:45.816939 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:45.817057 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:45.818297 (+ 1240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:45.818304 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:45.818804 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:45.819000 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:45.819013 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:45.820465 (+ 1452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:45.820469 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:45.821938 (+ 1469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:45.821954 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:45.823010 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:45.823016 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:45.823223 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:45.823849 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:30:45.823878 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:45.827013 (+ 3135us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.239.254:35823: Server oryx/127.28.239.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":402,"threads_started":1}
W20260811 06:30:45.827529 11783 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:30:45.816395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.195:43069 (local address 127.28.239.254:35823)
0811 06:30:45.817992 (+ 1597us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:45.818000 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:45.818018 (+ 18us) server_negotiation.cc:407] Connection header received
0811 06:30:45.818065 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:45.818070 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:45.818131 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:45.818210 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:45.819152 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:45.820312 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:45.822118 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:45.822822 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:45.822948 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:45.827222 (+ 4274us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.239.195:43069: BlockingRecv error: recv got EOF from 127.28.239.195:43069 (error 108)
Metrics: {"server-negotiator.queue_time_us":1464,"thread_start_us":1321,"threads_started":1}
Aug 11 06:30:46 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.194@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:30:46 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.194@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:30:46.301739 11785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:46.292430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:35823 (local address 127.28.239.194:47941)
0811 06:30:46.293154 (+ 724us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:46.293170 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:46.293289 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:46.293671 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:46.293674 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:46.294207 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:46.294383 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:46.294392 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:46.295477 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:46.295481 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:46.296464 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:46.296477 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:46.297224 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:46.297228 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:46.297370 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:46.298068 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:30:46.298093 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:46.301355 (+ 3262us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.239.254:35823: Server oryx/127.28.239.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":598,"thread_start_us":335,"threads_started":1}
W20260811 06:30:46.301949 11783 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:30:46.292544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.194:47941 (local address 127.28.239.254:35823)
0811 06:30:46.292749 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:46.292754 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:46.293245 (+ 491us) server_negotiation.cc:407] Connection header received
0811 06:30:46.293397 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:46.293401 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:46.293469 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:46.293682 (+ 213us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:46.294523 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:46.295304 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:46.296605 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:46.297042 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:46.297140 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:46.301663 (+ 4523us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.239.194:47941: BlockingRecv error: recv got EOF from 127.28.239.194:47941 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 11 06:30:46 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.193@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:30:46 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.195@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:30:46 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.193@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:30:46 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.195@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:30:46.849777 11786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:46.828174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:35823 (local address 127.28.239.193:45787)
0811 06:30:46.840454 (+ 12280us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:46.840481 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:46.840668 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:46.841205 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:46.841210 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:46.841869 (+ 659us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:46.842058 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:46.842068 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:46.843342 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:46.843348 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:46.844532 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:46.844546 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:46.845525 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:46.845556 (+ 31us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:46.845729 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:46.846453 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:30:46.846472 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:46.849454 (+ 2982us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.239.254:35823: Server oryx/127.28.239.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":12084,"thread_start_us":11990,"threads_started":1}
W20260811 06:30:46.850420 11791 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:30:46.828301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.193:45787 (local address 127.28.239.254:35823)
0811 06:30:46.840222 (+ 11921us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:46.840228 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:46.840827 (+ 599us) server_negotiation.cc:407] Connection header received
0811 06:30:46.840902 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:46.840910 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:46.841000 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:46.841108 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:46.842202 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:46.843191 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:46.844680 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:46.845337 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:46.845441 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:46.849741 (+ 4300us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.239.193:45787: BlockingRecv error: recv got EOF from 127.28.239.193:45787 (error 108)
Metrics: {"server-negotiator.queue_time_us":11776,"thread_start_us":12274,"threads_started":1}
I20260811 06:30:46.850554 11789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:46.828496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:35823 (local address 127.28.239.195:44091)
0811 06:30:46.841223 (+ 12727us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:46.841244 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:46.841397 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:46.841807 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:46.841812 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:46.842412 (+ 600us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:46.842633 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:46.842647 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:46.844018 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:46.844023 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:46.845202 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:46.845217 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:46.846260 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:46.846268 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:46.846425 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:46.846973 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:30:46.846990 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:46.850210 (+ 3220us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.239.254:35823: Server oryx/127.28.239.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":12600,"thread_start_us":12376,"threads_started":1}
W20260811 06:30:46.850790 11787 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:30:46.828619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.195:44091 (local address 127.28.239.254:35823)
0811 06:30:46.840825 (+ 12206us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:46.840830 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:46.841341 (+ 511us) server_negotiation.cc:407] Connection header received
0811 06:30:46.841533 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:46.841538 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:46.841642 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:46.841764 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:46.842788 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:46.843867 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:46.845354 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:46.846095 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:46.846187 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:46.850489 (+ 4302us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.239.195:44091: BlockingRecv error: recv got EOF from 127.28.239.195:44091 (error 108)
Metrics: {"server-negotiator.queue_time_us":12106,"thread_start_us":11327,"threads_started":1}
Aug 11 06:30:47 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.194@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:30:47 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.194@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:30:47.311800 11792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:47.302730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:35823 (local address 127.28.239.194:42723)
0811 06:30:47.303302 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:47.303319 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:47.303438 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:47.303776 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:47.303781 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:47.304213 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:47.304374 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:47.304383 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:47.305649 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:47.305657 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:47.306603 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:47.306615 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:47.307386 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:47.307393 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:47.307551 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:47.308208 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:30:47.308226 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:47.311443 (+ 3217us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.239.254:35823: Server oryx/127.28.239.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":302,"threads_started":1}
W20260811 06:30:47.311896 11787 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:30:47.302790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.194:42723 (local address 127.28.239.254:35823)
0811 06:30:47.302949 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:47.302954 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:47.303386 (+ 432us) server_negotiation.cc:407] Connection header received
0811 06:30:47.303545 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:47.303548 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:47.303611 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:47.303719 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:47.304520 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:47.305477 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:47.306746 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:47.307179 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:47.307317 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:47.311656 (+ 4339us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.239.194:42723: BlockingRecv error: recv got EOF from 127.28.239.194:42723 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 11 06:30:47 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.195@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:30:47 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.193@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:30:47 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.195@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:30:47 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.193@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:30:47.862638 11799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:47.851597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:35823 (local address 127.28.239.195:43975)
0811 06:30:47.852291 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:47.852316 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:47.852698 (+ 382us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:47.853131 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:47.853136 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:47.853753 (+ 617us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:47.854285 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:47.854303 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:47.855773 (+ 1470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:47.855777 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:47.856732 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:47.856754 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:47.857690 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:47.857695 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:47.857816 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:47.858653 (+ 837us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:30:47.858669 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:47.862247 (+ 3578us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.239.254:35823: Server oryx/127.28.239.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":363,"threads_started":1}
I20260811 06:30:47.862799 11795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:47.850984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:35823 (local address 127.28.239.193:54367)
0811 06:30:47.851610 (+ 626us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:47.851631 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:47.851803 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:47.853113 (+ 1310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:47.853118 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:47.853753 (+ 635us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:47.854017 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:47.854033 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:47.855361 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:47.855368 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:47.856557 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:47.856578 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:47.857429 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:47.857437 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:47.857697 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:47.858490 (+ 793us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:30:47.858520 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:47.862397 (+ 3877us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.239.254:35823: Server oryx/127.28.239.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":336,"threads_started":1}
W20260811 06:30:47.862859 11802 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:30:47.851965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.195:43975 (local address 127.28.239.254:35823)
0811 06:30:47.852664 (+ 699us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:47.852672 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:47.852696 (+ 24us) server_negotiation.cc:407] Connection header received
0811 06:30:47.852838 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:47.852844 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:47.852920 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:47.852996 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:47.854436 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:47.855616 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:47.856881 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:47.857498 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:47.857660 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:47.862480 (+ 4820us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.239.195:43975: BlockingRecv error: recv got EOF from 127.28.239.195:43975 (error 108)
Metrics: {"server-negotiator.queue_time_us":564,"thread_start_us":291,"threads_started":1}
W20260811 06:30:47.863094 11797 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:30:47.851100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.193:54367 (local address 127.28.239.254:35823)
0811 06:30:47.852534 (+ 1434us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:47.852541 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:47.852563 (+ 22us) server_negotiation.cc:407] Connection header received
0811 06:30:47.852643 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:47.852648 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:47.852762 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:47.852883 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:47.854203 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:47.855171 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:47.856717 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:47.857261 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:47.857353 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:47.862663 (+ 5310us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.239.193:54367: BlockingRecv error: recv got EOF from 127.28.239.193:54367 (error 108)
Metrics: {"server-negotiator.queue_time_us":1298,"thread_start_us":628,"threads_started":1}
Aug 11 06:30:48 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.194@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:30:48 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.194@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:30:48.327226 11803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:48.312892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:35823 (local address 127.28.239.194:57033)
0811 06:30:48.313439 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:48.313455 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:48.313628 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:48.313964 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:48.313967 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:48.314373 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:48.314541 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:48.314550 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:48.315575 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:48.315579 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:48.316619 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:48.316631 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:48.317404 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:48.317408 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:48.317525 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:48.323871 (+ 6346us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:30:48.323890 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:48.326965 (+ 3075us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.239.254:35823: Server oryx/127.28.239.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":323,"threads_started":1}
W20260811 06:30:48.327509 11797 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:30:48.312957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.194:57033 (local address 127.28.239.254:35823)
0811 06:30:48.313215 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:48.313225 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:48.313553 (+ 328us) server_negotiation.cc:407] Connection header received
0811 06:30:48.313742 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:48.313746 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:48.313825 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:48.313909 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:48.314683 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:48.315468 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:48.316770 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:48.317266 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:48.317363 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:48.327258 (+ 9895us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.239.194:57033: BlockingRecv error: recv got EOF from 127.28.239.194:57033 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 11 06:30:48 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.195@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:30:48 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.193@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:30:48 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.195@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:30:48 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.193@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:30:48.889297 11804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:48.864199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:35823 (local address 127.28.239.195:56141)
0811 06:30:48.879300 (+ 15101us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:48.879333 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:48.879529 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:48.880609 (+ 1080us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:48.880615 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:48.881193 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:48.881446 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:48.881462 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:48.882796 (+ 1334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:48.882802 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:48.883771 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:48.883783 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:48.884500 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:48.884506 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:48.884685 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:48.885383 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:30:48.885410 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:48.888936 (+ 3526us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.239.254:35823: Server oryx/127.28.239.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":14879,"thread_start_us":15829,"threads_started":1}
W20260811 06:30:48.889461 11808 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:30:48.864249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.195:56141 (local address 127.28.239.254:35823)
0811 06:30:48.880175 (+ 15926us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:48.880182 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:48.880199 (+ 17us) server_negotiation.cc:407] Connection header received
0811 06:30:48.880264 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:48.880271 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:48.880367 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:48.880494 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:48.881775 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:48.882654 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:48.883910 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:48.884333 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:48.884471 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:48.889160 (+ 4689us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.239.195:56141: BlockingRecv error: recv got EOF from 127.28.239.195:56141 (error 108)
Metrics: {"server-negotiator.queue_time_us":15776,"spinlock_wait_cycles":44416,"thread_start_us":15635,"threads_started":1}
I20260811 06:30:48.889672 11810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:48.864240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:35823 (local address 127.28.239.193:54485)
0811 06:30:48.879338 (+ 15098us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:48.879371 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:48.879563 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:48.880865 (+ 1302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:48.880870 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:48.881412 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:48.881626 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:48.881643 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:48.882983 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:48.882994 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:48.884023 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:48.884046 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:48.884903 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:48.884910 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:48.885121 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:48.885739 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:30:48.885766 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:48.889319 (+ 3553us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.239.254:35823: Server oryx/127.28.239.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":14873,"thread_start_us":15766,"threads_started":1}
W20260811 06:30:48.889972 11812 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:30:48.864268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.193:54485 (local address 127.28.239.254:35823)
0811 06:30:48.880444 (+ 16176us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:48.880452 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:48.880475 (+ 23us) server_negotiation.cc:407] Connection header received
0811 06:30:48.880531 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:48.880536 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:48.880652 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:48.880783 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:48.881814 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:48.882812 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:48.884209 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:48.884724 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:48.884870 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:48.889611 (+ 4741us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.239.193:54485: BlockingRecv error: recv got EOF from 127.28.239.193:54485 (error 108)
Metrics: {"server-negotiator.queue_time_us":16032,"spinlock_wait_cycles":896,"thread_start_us":15599,"threads_started":1}
Aug 11 06:30:49 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.194@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:30:49 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.194@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:30:49.336731 11815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:49.328258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:35823 (local address 127.28.239.194:35081)
0811 06:30:49.328846 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:49.328862 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:49.328978 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:49.329311 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:49.329314 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:49.329699 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:49.329871 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:49.329880 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:49.330967 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:49.330970 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:49.331910 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:49.331920 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:49.332660 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:49.332664 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:49.332791 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:49.333451 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:30:49.333469 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:49.336493 (+ 3024us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.239.254:35823: Server oryx/127.28.239.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":346,"threads_started":1}
W20260811 06:30:49.336941 11812 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:30:49.328365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.194:35081 (local address 127.28.239.254:35823)
0811 06:30:49.328639 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:49.328647 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:49.328948 (+ 301us) server_negotiation.cc:407] Connection header received
0811 06:30:49.329108 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:49.329113 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:49.329181 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:49.329268 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:49.330036 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:49.330842 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:49.332041 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:49.332516 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:49.332631 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:49.336710 (+ 4079us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.239.194:35081: BlockingRecv error: recv got EOF from 127.28.239.194:35081 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Aug 11 06:30:49 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.193@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:30:49 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.195@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:30:49 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.193@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:30:49 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.195@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:30:49.914546 11820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:49.890757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:35823 (local address 127.28.239.193:40925)
0811 06:30:49.902877 (+ 12120us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:49.902904 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:49.903078 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:49.903626 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:49.903635 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:49.904262 (+ 627us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:49.905102 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:49.905124 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:49.906904 (+ 1780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:49.906910 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:49.908298 (+ 1388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:49.908310 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:49.909278 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:49.909284 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:49.909469 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:49.910340 (+ 871us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:30:49.910358 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:49.914177 (+ 3819us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.239.254:35823: Server oryx/127.28.239.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":11992,"thread_start_us":11850,"threads_started":1}
W20260811 06:30:49.914691 11821 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:30:49.890959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.193:40925 (local address 127.28.239.254:35823)
0811 06:30:49.901281 (+ 10322us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:49.901290 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:49.902982 (+ 1692us) server_negotiation.cc:407] Connection header received
0811 06:30:49.903281 (+ 299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:49.903289 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:49.903415 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:49.903544 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:49.905330 (+ 1786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:49.906741 (+ 1411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:49.908440 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:49.909110 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:49.909287 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:49.914375 (+ 5088us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.239.193:40925: BlockingRecv error: recv got EOF from 127.28.239.193:40925 (error 108)
Metrics: {"mutex_wait_us":137,"server-negotiator.queue_time_us":10159,"thread_start_us":10212,"threads_started":1}
I20260811 06:30:49.915844 11819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:49.890732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:35823 (local address 127.28.239.195:51715)
0811 06:30:49.903223 (+ 12491us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:49.903245 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:49.903385 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:49.904509 (+ 1124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:49.904513 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:49.905024 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:49.905287 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:49.905305 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:49.906519 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:49.906525 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:49.908031 (+ 1506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:49.908055 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:49.909497 (+ 1442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:49.909502 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:49.909879 (+ 377us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:49.910568 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:30:49.910591 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:49.915492 (+ 4901us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.239.254:35823: Server oryx/127.28.239.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":12354,"thread_start_us":12261,"threads_started":1}
W20260811 06:30:49.916893 11822 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:30:49.897011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.195:51715 (local address 127.28.239.254:35823)
0811 06:30:49.904163 (+ 7152us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:49.904170 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:49.904189 (+ 19us) server_negotiation.cc:407] Connection header received
0811 06:30:49.904262 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:49.904269 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:49.904349 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:49.904473 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:49.905516 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:49.906383 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:49.908210 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:49.909283 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:49.909430 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:49.916445 (+ 7015us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.239.195:51715: BlockingRecv error: recv got EOF from 127.28.239.195:51715 (error 108)
Metrics: {"mutex_wait_us":43,"server-negotiator.queue_time_us":7001,"thread_start_us":6840,"threads_started":1}
Aug 11 06:30:50 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.194@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:30:50 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.194@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:30:50.346402 11823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:50.337866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:35823 (local address 127.28.239.194:51943)
0811 06:30:50.338355 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:50.338372 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:50.338492 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:50.338798 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:50.338802 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:50.339245 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:50.339421 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:50.339429 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:50.340480 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:50.340483 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:50.341381 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:50.341392 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:50.342138 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:50.342144 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:50.342296 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:50.342914 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:30:50.342932 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:50.346156 (+ 3224us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.239.254:35823: Server oryx/127.28.239.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":268,"threads_started":1}
W20260811 06:30:50.346730 11822 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:30:50.337946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.194:51943 (local address 127.28.239.254:35823)
0811 06:30:50.338181 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:50.338188 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:50.338453 (+ 265us) server_negotiation.cc:407] Connection header received
0811 06:30:50.338597 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:50.338602 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:50.338668 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:50.338750 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:50.339586 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:50.340358 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:50.341550 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:50.341983 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:50.342078 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:50.346353 (+ 4275us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.239.194:51943: BlockingRecv error: recv got EOF from 127.28.239.194:51943 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 11 06:30:50 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.193@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:30:50 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.193@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:30:50.926769 11826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:50.915600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:35823 (local address 127.28.239.193:57563)
0811 06:30:50.916302 (+ 702us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:50.916320 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:50.916437 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:50.916836 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:50.916841 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:50.917507 (+ 666us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:50.917809 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:50.917825 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:50.918987 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:50.918991 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:50.920109 (+ 1118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:50.920145 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:50.921282 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:50.921289 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:50.921500 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:50.922569 (+ 1069us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:30:50.922594 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:50.926423 (+ 3829us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.239.254:35823: Server oryx/127.28.239.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":426,"threads_started":1}
W20260811 06:30:50.926954 11828 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:30:50.915725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.193:57563 (local address 127.28.239.254:35823)
0811 06:30:50.916404 (+ 679us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:50.916409 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:50.916429 (+ 20us) server_negotiation.cc:407] Connection header received
0811 06:30:50.916536 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:50.916541 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:50.916644 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:50.916774 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:50.917987 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:50.918841 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:50.920282 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:50.921101 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:50.921229 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:50.926664 (+ 5435us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.239.193:57563: BlockingRecv error: recv got EOF from 127.28.239.193:57563 (error 108)
Metrics: {"mutex_wait_us":46,"server-negotiator.queue_time_us":568,"thread_start_us":433,"threads_started":1}
Aug 11 06:30:50 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.195@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:30:50 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.195@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:30:50.928805 11831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:50.917082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:35823 (local address 127.28.239.195:39331)
0811 06:30:50.917710 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:50.917733 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:50.917896 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:50.919401 (+ 1505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:50.919405 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:50.919849 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:50.920038 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:50.920049 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:50.921709 (+ 1660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:50.921717 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:50.923560 (+ 1843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:50.923575 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:50.924508 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:50.924515 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:50.924725 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:50.925468 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:30:50.925494 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:50.928566 (+ 3072us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.239.254:35823: Server oryx/127.28.239.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":290,"threads_started":1}
W20260811 06:30:50.929137 11835 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:30:50.917198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.195:39331 (local address 127.28.239.254:35823)
0811 06:30:50.917953 (+ 755us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:50.917960 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:50.917979 (+ 19us) server_negotiation.cc:407] Connection header received
0811 06:30:50.919108 (+ 1129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:50.919113 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:50.919201 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:50.919325 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:50.920220 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:50.921513 (+ 1293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:50.923706 (+ 2193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:50.924303 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:50.924438 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:50.928795 (+ 4357us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.239.195:39331: BlockingRecv error: recv got EOF from 127.28.239.195:39331 (error 108)
Metrics: {"mutex_wait_us":10,"server-negotiator.queue_time_us":621,"thread_start_us":471,"threads_started":1}
Aug 11 06:30:51 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.194@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:30:51 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.194@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:30:51.356050 11838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:51.347417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:35823 (local address 127.28.239.194:39951)
0811 06:30:51.347946 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:51.347962 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:51.348119 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:51.348501 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:51.348506 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:51.348944 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:51.349104 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:51.349113 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:51.350259 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:51.350268 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:51.351227 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:51.351239 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:51.351925 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:51.351931 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:51.352113 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:51.352717 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:30:51.352735 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:51.355806 (+ 3071us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.239.254:35823: Server oryx/127.28.239.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":308,"threads_started":1}
W20260811 06:30:51.356408 11835 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:30:51.347470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.194:39951 (local address 127.28.239.254:35823)
0811 06:30:51.347694 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:51.347703 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:51.348047 (+ 344us) server_negotiation.cc:407] Connection header received
0811 06:30:51.348258 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:51.348263 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:51.348340 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:51.348443 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:51.349246 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:51.350108 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:51.351366 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:51.351781 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:51.351910 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:51.356076 (+ 4166us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.239.194:39951: BlockingRecv error: recv got EOF from 127.28.239.194:39951 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 11 06:30:51 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.193@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:30:51 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.195@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:30:51 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.193@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:30:51 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.195@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:30:51.939697 11839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:51.927950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:35823 (local address 127.28.239.193:60021)
0811 06:30:51.928483 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:51.928500 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:51.928611 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:51.929188 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:51.929191 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:51.929820 (+ 629us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:51.930119 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:51.930141 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:51.931734 (+ 1593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:51.931742 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:51.933032 (+ 1290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:51.933057 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:51.933977 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:51.933987 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:51.934197 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:51.935043 (+ 846us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:30:51.935103 (+ 60us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:51.939327 (+ 4224us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.239.254:35823: Server oryx/127.28.239.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":275,"threads_started":1}
W20260811 06:30:51.939877 11840 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:30:51.928043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.193:60021 (local address 127.28.239.254:35823)
0811 06:30:51.928887 (+ 844us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:51.928892 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:51.928909 (+ 17us) server_negotiation.cc:407] Connection header received
0811 06:30:51.928967 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:51.928971 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:51.929038 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:51.929141 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:51.930340 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:51.931553 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:51.933228 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:51.933758 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:51.933858 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:51.939572 (+ 5714us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.239.193:60021: BlockingRecv error: recv got EOF from 127.28.239.193:60021 (error 108)
Metrics: {"server-negotiator.queue_time_us":737,"thread_start_us":573,"threads_started":1}
I20260811 06:30:51.940279 11841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:51.929845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:35823 (local address 127.28.239.195:57197)
0811 06:30:51.930471 (+ 626us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:51.930498 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:51.930657 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:51.931273 (+ 616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:51.931277 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:51.931775 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:51.932008 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:51.932022 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:51.933120 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:51.933129 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:51.934625 (+ 1496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:51.934640 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:51.935478 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:51.935485 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:51.935705 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:51.936407 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:30:51.936425 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:51.939931 (+ 3506us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.239.254:35823: Server oryx/127.28.239.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":335,"threads_started":1}
W20260811 06:30:51.940481 11843 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:30:51.929941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.195:57197 (local address 127.28.239.254:35823)
0811 06:30:51.930867 (+ 926us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:51.930874 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:51.930893 (+ 19us) server_negotiation.cc:407] Connection header received
0811 06:30:51.930958 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:51.930963 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:51.931035 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:51.931149 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:51.932174 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:51.932949 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:51.934799 (+ 1850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:51.935274 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:51.935371 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:51.940157 (+ 4786us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.239.195:57197: BlockingRecv error: recv got EOF from 127.28.239.195:57197 (error 108)
Metrics: {"server-negotiator.queue_time_us":775,"thread_start_us":573,"threads_started":1}
Aug 11 06:30:52 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.194@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:30:52 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.194@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:30:52.366356 11844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:52.357094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:35823 (local address 127.28.239.194:43979)
0811 06:30:52.357725 (+ 631us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:52.357741 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:52.357855 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:52.358279 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:52.358287 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:52.358780 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:52.358974 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:52.358984 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:52.360286 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:52.360292 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:52.361272 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:52.361285 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:52.362283 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:52.362288 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:52.362411 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:52.363036 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:30:52.363054 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:52.366000 (+ 2946us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.239.254:35823: Server oryx/127.28.239.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":334,"threads_started":1}
W20260811 06:30:52.366472 11843 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:30:52.357253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.194:43979 (local address 127.28.239.254:35823)
0811 06:30:52.357461 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:52.357469 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:52.357809 (+ 340us) server_negotiation.cc:407] Connection header received
0811 06:30:52.358000 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:52.358006 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:52.358098 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:52.358203 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:52.359183 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:52.360070 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:52.361414 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:52.362099 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:52.362207 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:52.366208 (+ 4001us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.239.194:43979: BlockingRecv error: recv got EOF from 127.28.239.194:43979 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 11 06:30:52 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.193@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:30:52 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.193@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:30:52 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.195@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:30:52.951920 11846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:52.940951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:35823 (local address 127.28.239.193:56117)
0811 06:30:52.941641 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:52.941674 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:52.941879 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:52.943343 (+ 1464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:52.943348 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:52.943745 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:52.943931 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:52.943941 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:52.945012 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:52.945018 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:52.946029 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:52.946042 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:52.946755 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:52.946761 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:52.946934 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:52.947623 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:30:52.947647 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:52.951547 (+ 3900us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.239.254:35823: Server oryx/127.28.239.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":285,"threads_started":1}
W20260811 06:30:52.952092 11852 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:30:52.941100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.193:56117 (local address 127.28.239.254:35823)
0811 06:30:52.942906 (+ 1806us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:52.942913 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:52.942940 (+ 27us) server_negotiation.cc:407] Connection header received
0811 06:30:52.943014 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:52.943021 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:52.943139 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:52.943275 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:52.944092 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:52.944843 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:52.946179 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:52.946585 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:52.946724 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:52.951832 (+ 5108us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.239.193:56117: BlockingRecv error: recv got EOF from 127.28.239.193:56117 (error 108)
Metrics: {"server-negotiator.queue_time_us":1638,"thread_start_us":1723,"threads_started":1}
Aug 11 06:30:52 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.195@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:30:52.953233 11849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:52.941376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:35823 (local address 127.28.239.195:53687)
0811 06:30:52.943355 (+ 1979us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:52.943377 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:52.943525 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:52.943905 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:52.943912 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:52.944402 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:52.944584 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:52.944594 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:52.945909 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:52.945914 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:52.947142 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:52.947169 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:52.948244 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:52.948249 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:52.948450 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:52.949221 (+ 771us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:30:52.949247 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:52.952924 (+ 3677us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.239.254:35823: Server oryx/127.28.239.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1859,"thread_start_us":1713,"threads_started":1}
W20260811 06:30:52.953346 11847 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:30:52.941771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.195:53687 (local address 127.28.239.254:35823)
0811 06:30:52.943161 (+ 1390us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:52.943167 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:52.943453 (+ 286us) server_negotiation.cc:407] Connection header received
0811 06:30:52.943672 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:52.943684 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:52.943739 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:52.943860 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:52.944751 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:52.945748 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:52.947346 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:52.948056 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:52.948192 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:52.953097 (+ 4905us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.239.195:53687: BlockingRecv error: recv got EOF from 127.28.239.195:53687 (error 108)
Metrics: {"server-negotiator.queue_time_us":1258,"thread_start_us":732,"threads_started":1}
Aug 11 06:30:53 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.194@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:30:53 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.194@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:30:53.379902 11854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:53.367515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:35823 (local address 127.28.239.194:49205)
0811 06:30:53.368076 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:53.368098 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:53.368259 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:53.368715 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:53.368719 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:53.369223 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:53.369425 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:53.369435 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:53.370602 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:53.370605 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:53.371536 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:53.371548 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:53.372300 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:53.372304 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:53.372434 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:53.376308 (+ 3874us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:30:53.376329 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:53.379553 (+ 3224us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.239.254:35823: Server oryx/127.28.239.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":288,"threads_started":1}
W20260811 06:30:53.380460 11847 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:30:53.367593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.194:49205 (local address 127.28.239.254:35823)
0811 06:30:53.367857 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:53.367864 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:53.368194 (+ 330us) server_negotiation.cc:407] Connection header received
0811 06:30:53.368424 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:53.368430 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:53.368521 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:53.368644 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:53.369614 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:53.370465 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:53.371708 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:53.372136 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:53.372240 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:53.379907 (+ 7667us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.239.194:49205: BlockingRecv error: recv got EOF from 127.28.239.194:49205 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Aug 11 06:30:53 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.195@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:30:53 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.193@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:30:53 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.195@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:30:53 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.193@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:30:53.963863 11858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:53.954381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:35823 (local address 127.28.239.195:47421)
0811 06:30:53.954886 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:53.954906 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:53.955039 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:53.955551 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:53.955558 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:53.955949 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:53.956177 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:53.956191 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:53.957328 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:53.957334 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:53.958342 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:53.958354 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:53.959473 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:53.959479 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:53.959682 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:53.960217 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:30:53.960241 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:53.963639 (+ 3398us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.239.254:35823: Server oryx/127.28.239.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":272,"threads_started":1}
W20260811 06:30:53.964244 11859 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:30:53.954441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.195:47421 (local address 127.28.239.254:35823)
0811 06:30:53.955224 (+ 783us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:53.955228 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:53.955241 (+ 13us) server_negotiation.cc:407] Connection header received
0811 06:30:53.955323 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:53.955330 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:53.955399 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:53.955509 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:53.956327 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:53.957184 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:53.958487 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:53.959342 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:53.959458 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:53.963906 (+ 4448us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.239.195:47421: BlockingRecv error: recv got EOF from 127.28.239.195:47421 (error 108)
Metrics: {"server-negotiator.queue_time_us":671,"thread_start_us":466,"threads_started":1}
I20260811 06:30:53.964331 11856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:53.953279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:35823 (local address 127.28.239.193:52225)
0811 06:30:53.953895 (+ 616us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:53.953913 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:53.954051 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:53.954879 (+ 828us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:53.954888 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:53.955483 (+ 595us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:53.955681 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:53.955692 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:53.957019 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:53.957025 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:53.958171 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:53.958184 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:53.958987 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:53.958994 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:53.959202 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:53.960073 (+ 871us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:30:53.960143 (+ 70us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:53.963986 (+ 3843us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.239.254:35823: Server oryx/127.28.239.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":354,"threads_started":1}
W20260811 06:30:53.964833 11857 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:30:53.953346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.193:52225 (local address 127.28.239.254:35823)
0811 06:30:53.954211 (+ 865us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:53.954216 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:53.954236 (+ 20us) server_negotiation.cc:407] Connection header received
0811 06:30:53.954293 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:53.954297 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:53.954427 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:53.954566 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:53.955859 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:53.956840 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:53.958330 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:53.958812 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:53.958970 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:53.964586 (+ 5616us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.239.193:52225: BlockingRecv error: recv got EOF from 127.28.239.193:52225 (error 108)
Metrics: {"server-negotiator.queue_time_us":631,"thread_start_us":648,"threads_started":1}
Aug 11 06:30:54 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.194@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:30:54 dist-test-slave-gkw7 krb5kdc[10228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.239.194@KRBTEST.COM for oryx/127.28.239.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:30:54.389343 11860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:54.381130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:35823 (local address 127.28.239.194:47063)
0811 06:30:54.381739 (+ 609us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:54.381770 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:54.381904 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:54.382222 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:54.382225 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:54.382618 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:54.382806 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:54.382815 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:54.383844 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:54.383847 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:54.384777 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:54.384788 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:54.385460 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:54.385464 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:54.385592 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:54.386241 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:30:54.386262 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:54.389090 (+ 2828us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.239.254:35823: Server oryx/127.28.239.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":326,"threads_started":1}
W20260811 06:30:54.389533 11857 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:30:54.381245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.194:47063 (local address 127.28.239.254:35823)
0811 06:30:54.381434 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:54.381439 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:54.381870 (+ 431us) server_negotiation.cc:407] Connection header received
0811 06:30:54.382020 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:54.382024 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:54.382092 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:54.382192 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:54.382946 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:54.383705 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:54.384913 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:54.385337 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:54.385432 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:54.389292 (+ 3860us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.239.194:47063: BlockingRecv error: recv got EOF from 127.28.239.194:47063 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 06:30:54.723002 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 11105
I20260811 06:30:54.736683 11864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:54.724244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38022 (local address 127.28.239.193:43093)
0811 06:30:54.724941 (+ 697us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:54.724949 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:54.724976 (+ 27us) server_negotiation.cc:407] Connection header received
0811 06:30:54.725058 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:54.725064 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:54.725316 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:54.725469 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:54.727088 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:54.728175 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:54.729436 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:54.730075 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:54.730175 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:54.732835 (+ 2660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:54.732847 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:54.732864 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:30:54.732902 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:54.735909 (+ 3007us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.28.239.193@KRBTEST.COM not found in keytab (ticket kvno 2)
0811 06:30:54.736435 (+ 526us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:38022: Request ticket server kudu/127.28.239.193@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":524,"thread_start_us":382,"threads_started":1}
W20260811 06:30:54.736827 11864 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:38022: Request ticket server kudu/127.28.239.193@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260811 06:30:54.736905 29631 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 11105: Not authorized: Client connection negotiation failed: client connection to 127.28.239.193:43093: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.28.239.193@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260811 06:30:54.737174 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 11105
I20260811 06:30:54.766952 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 11347
I20260811 06:30:54.779649 11865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:54.768096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41988 (local address 127.28.239.194:44689)
0811 06:30:54.768680 (+ 584us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:54.768687 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:54.768721 (+ 34us) server_negotiation.cc:407] Connection header received
0811 06:30:54.768790 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:54.768797 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:54.769022 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:54.769177 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:54.770578 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:54.771666 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:54.772900 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:54.773476 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:54.773780 (+ 304us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:54.776375 (+ 2595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:54.776385 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:54.776402 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:30:54.776437 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:54.778899 (+ 2462us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.28.239.194@KRBTEST.COM not found in keytab (ticket kvno 2)
0811 06:30:54.779259 (+ 360us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41988: Request ticket server kudu/127.28.239.194@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":429,"thread_start_us":276,"threads_started":1}
W20260811 06:30:54.779654 29631 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 11347: Not authorized: Client connection negotiation failed: client connection to 127.28.239.194:44689: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.28.239.194@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260811 06:30:54.779767 11865 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41988: Request ticket server kudu/127.28.239.194@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260811 06:30:54.779815 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 11347
I20260811 06:30:54.809737 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 11543
I20260811 06:30:54.823335 11866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:54.810834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53116 (local address 127.28.239.195:42883)
0811 06:30:54.811495 (+ 661us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:54.811503 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:54.811532 (+ 29us) server_negotiation.cc:407] Connection header received
0811 06:30:54.811597 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:54.811603 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:54.811833 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:54.811999 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:54.813351 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:54.814600 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:54.815892 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:54.816692 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:54.816807 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:54.819521 (+ 2714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:54.819531 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:54.819564 (+ 33us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:30:54.819604 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:54.822740 (+ 3136us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.28.239.195@KRBTEST.COM not found in keytab (ticket kvno 2)
0811 06:30:54.823060 (+ 320us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:53116: Request ticket server kudu/127.28.239.195@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":501,"thread_start_us":328,"threads_started":1}
W20260811 06:30:54.823482 11866 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:53116: Request ticket server kudu/127.28.239.195@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260811 06:30:54.823572 29631 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 11543: Not authorized: Client connection negotiation failed: client connection to 127.28.239.195:42883: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.28.239.195@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260811 06:30:54.823771 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 11543
I20260811 06:30:54.850548 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 11007
I20260811 06:30:54.851917 11097 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:30:54.971897 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 11007
2026-08-11T06:30:55Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (21866 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-11T06:30:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:30:55Z Disabled control of system clock
I20260811 06:30:55.078258 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429755927508-29631-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429755927508-29631-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429755927508-29631-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:40043
--webserver_interface=127.28.239.254
--webserver_port=0
--builtin_ntp_servers=127.28.239.212:46267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.239.254:40043
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 06:30:55.439136 11872 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:30:55.439472 11872 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:30:55.439634 11872 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:30:55.447237 11872 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:30:55.447384 11872 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:30:55.447481 11872 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:30:55.447589 11872 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:30:55.447665 11872 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:30:55.457113 11872 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:46267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429755927508-29631-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429755927508-29631-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.239.254:40043
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:40043
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.239.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429755927508-29631-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:30:55.458581 11872 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:30:55.460367 11872 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:30:55.473038 11885 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:30:55.473003 11887 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:30:55.473120 11884 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:30:55.479446 11872 server_base.cc:1034] running on GCE node
I20260811 06:30:55.480459 11872 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:30:55.482297 11872 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:30:55.483543 11872 hybrid_clock.cc:648] HybridClock initialized: now 1786429855483487 us; error 52 us; skew 500 ppm
I20260811 06:30:55.483920 11872 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:30:55.486377 11872 webserver.cc:460] Webserver started at http://127.28.239.254:33767/ using document root <none> and password file <none>
I20260811 06:30:55.487124 11872 fs_manager.cc:359] Metadata directory not provided
I20260811 06:30:55.487282 11872 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:30:55.487633 11872 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:30:55.490931 11872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429755927508-29631-0/minicluster-data/master-0/data/instance:
uuid: "5c4dda9d594b467d83084ce960704800"
format_stamp: "Formatted at 2026-08-11 06:30:55 on dist-test-slave-gkw7"
server_key: "3bc4a7cb0c893db440646c9e5cc78f09"
server_key_iv: "2a6117539c23914e3bb532303bb79edd"
server_key_version: "encryptionkey@0"
I20260811 06:30:55.491762 11872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429755927508-29631-0/minicluster-data/master-0/wal/instance:
uuid: "5c4dda9d594b467d83084ce960704800"
format_stamp: "Formatted at 2026-08-11 06:30:55 on dist-test-slave-gkw7"
server_key: "3bc4a7cb0c893db440646c9e5cc78f09"
server_key_iv: "2a6117539c23914e3bb532303bb79edd"
server_key_version: "encryptionkey@0"
I20260811 06:30:55.497169 11872 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 06:30:55.500813 11896 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:55.502343 11872 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:30:55.502544 11872 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429755927508-29631-0/minicluster-data/master-0/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429755927508-29631-0/minicluster-data/master-0/wal
uuid: "5c4dda9d594b467d83084ce960704800"
format_stamp: "Formatted at 2026-08-11 06:30:55 on dist-test-slave-gkw7"
server_key: "3bc4a7cb0c893db440646c9e5cc78f09"
server_key_iv: "2a6117539c23914e3bb532303bb79edd"
server_key_version: "encryptionkey@0"
I20260811 06:30:55.502777 11872 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429755927508-29631-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429755927508-29631-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429755927508-29631-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:30:55.520551 11872 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:30:55.521498 11872 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:30:55.521899 11872 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:30:55.540666 11872 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.254:40043
I20260811 06:30:55.540674 11981 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.254:40043 every 8 connection(s)
I20260811 06:30:55.542151 11872 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
I20260811 06:30:55.545958 11982 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:30:55.549412 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 11872
I20260811 06:30:55.549644 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429755927508-29631-0/minicluster-data/master-0/wal/instance
I20260811 06:30:55.550518 29631 external_mini_cluster.cc:1428] Setting key 11ee8de126a3179e6a4e46b476eda523
I20260811 06:30:55.559824 11982 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c4dda9d594b467d83084ce960704800: Bootstrap starting.
I20260811 06:30:55.563735 11982 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5c4dda9d594b467d83084ce960704800: Neither blocks nor log segments found. Creating new log.
I20260811 06:30:55.565023 11982 log.cc:824] T 00000000000000000000000000000000 P 5c4dda9d594b467d83084ce960704800: Log is configured to *not* fsync() on all Append() calls
I20260811 06:30:55.566447 11985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:55.552860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51896 (local address 127.28.239.254:40043)
0811 06:30:55.553953 (+ 1093us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:55.553974 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:55.554021 (+ 47us) server_negotiation.cc:407] Connection header received
0811 06:30:55.555839 (+ 1818us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:55.555870 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:55.556468 (+ 598us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:55.556868 (+ 400us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:55.557644 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:55.559171 (+ 1527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:55.561045 (+ 1874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:55.561990 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:55.562163 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:55.563439 (+ 1276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:55.563448 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:55.563463 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:30:55.564169 (+ 706us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:55.564277 (+ 108us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:30:55.564301 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:55.564398 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:55.564682 (+ 284us) server_negotiation.cc:299] Negotiation successful
0811 06:30:55.565055 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":631,"thread_start_us":405,"threads_started":1}
I20260811 06:30:55.568493 11982 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c4dda9d594b467d83084ce960704800: No bootstrap required, opened a new log
I20260811 06:30:55.576237 11982 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5c4dda9d594b467d83084ce960704800 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c4dda9d594b467d83084ce960704800" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 40043 } }
I20260811 06:30:55.576635 11982 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5c4dda9d594b467d83084ce960704800 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:30:55.576759 11982 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5c4dda9d594b467d83084ce960704800 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c4dda9d594b467d83084ce960704800, State: Initialized, Role: FOLLOWER
I20260811 06:30:55.577379 11982 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5c4dda9d594b467d83084ce960704800 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c4dda9d594b467d83084ce960704800" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 40043 } }
I20260811 06:30:55.577675 11982 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5c4dda9d594b467d83084ce960704800 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:30:55.577836 11982 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5c4dda9d594b467d83084ce960704800 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:30:55.578018 11982 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5c4dda9d594b467d83084ce960704800 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:30:55.580193 11982 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5c4dda9d594b467d83084ce960704800 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c4dda9d594b467d83084ce960704800" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 40043 } }
I20260811 06:30:55.580718 11982 leader_election.cc:302] T 00000000000000000000000000000000 P 5c4dda9d594b467d83084ce960704800 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5c4dda9d594b467d83084ce960704800; no voters:
I20260811 06:30:55.581344 11982 leader_election.cc:288] T 00000000000000000000000000000000 P 5c4dda9d594b467d83084ce960704800 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:30:55.581584 11988 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5c4dda9d594b467d83084ce960704800 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:30:55.582842 11988 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5c4dda9d594b467d83084ce960704800 [term 1 LEADER]: Becoming Leader. State: Replica: 5c4dda9d594b467d83084ce960704800, State: Running, Role: LEADER
I20260811 06:30:55.583549 11982 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5c4dda9d594b467d83084ce960704800 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:30:55.583492 11988 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5c4dda9d594b467d83084ce960704800 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c4dda9d594b467d83084ce960704800" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 40043 } }
I20260811 06:30:55.586390 11990 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c4dda9d594b467d83084ce960704800 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5c4dda9d594b467d83084ce960704800. Latest consensus state: current_term: 1 leader_uuid: "5c4dda9d594b467d83084ce960704800" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c4dda9d594b467d83084ce960704800" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 40043 } } }
I20260811 06:30:55.586822 11990 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c4dda9d594b467d83084ce960704800 [sys.catalog]: This master's current role is: LEADER
I20260811 06:30:55.587363 11989 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c4dda9d594b467d83084ce960704800 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5c4dda9d594b467d83084ce960704800" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c4dda9d594b467d83084ce960704800" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 40043 } } }
I20260811 06:30:55.587699 11989 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c4dda9d594b467d83084ce960704800 [sys.catalog]: This master's current role is: LEADER
I20260811 06:30:55.589432 12002 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:30:55.594638 12002 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:30:55.604234 12002 catalog_manager.cc:1347] Generated new cluster ID: 1092b9812b904d0d8452ef54f89e077d
I20260811 06:30:55.604365 12002 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:30:55.633678 12002 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:30:55.635212 12002 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:30:55.643162 12002 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5c4dda9d594b467d83084ce960704800: Generated new TSK 0
I20260811 06:30:55.644078 12002 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:30:55.654538 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.193:0
--local_ip_for_outbound_sockets=127.28.239.193
--webserver_interface=127.28.239.193
--webserver_port=0
--tserver_master_addrs=127.28.239.254:40043
--builtin_ntp_servers=127.28.239.212:46267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260811 06:30:56.004524 12017 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:30:56.004796 12017 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:30:56.004882 12017 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:30:56.012413 12017 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:30:56.012514 12017 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:30:56.012686 12017 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.193
I20260811 06:30:56.021800 12017 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:46267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.239.193
--webserver_port=0
--tserver_master_addrs=127.28.239.254:40043
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.193
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:30:56.023190 12017 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:30:56.025084 12017 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:30:56.038816 12023 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:30:56.038771 12028 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:30:56.038856 12024 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:30:56.039976 12017 server_base.cc:1034] running on GCE node
I20260811 06:30:56.040581 12017 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:30:56.041503 12017 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:30:56.042791 12017 hybrid_clock.cc:648] HybridClock initialized: now 1786429856042747 us; error 44 us; skew 500 ppm
I20260811 06:30:56.043174 12017 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:30:56.045645 12017 webserver.cc:460] Webserver started at http://127.28.239.193:46193/ using document root <none> and password file <none>
I20260811 06:30:56.046427 12017 fs_manager.cc:359] Metadata directory not provided
I20260811 06:30:56.046569 12017 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:30:56.046932 12017 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:30:56.049969 12017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-0/data/instance:
uuid: "38482fb2b4114b81a9693eccc95c7f31"
format_stamp: "Formatted at 2026-08-11 06:30:56 on dist-test-slave-gkw7"
server_key: "a9fd85a73b131e6857c4f8d5e7217ba1"
server_key_iv: "d005d2eab4206d4aa21280ba3b31d4a2"
server_key_version: "encryptionkey@0"
I20260811 06:30:56.050807 12017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-0/wal/instance:
uuid: "38482fb2b4114b81a9693eccc95c7f31"
format_stamp: "Formatted at 2026-08-11 06:30:56 on dist-test-slave-gkw7"
server_key: "a9fd85a73b131e6857c4f8d5e7217ba1"
server_key_iv: "d005d2eab4206d4aa21280ba3b31d4a2"
server_key_version: "encryptionkey@0"
I20260811 06:30:56.056161 12017 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 06:30:56.059777 12040 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:56.061203 12017 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 06:30:56.061434 12017 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-0/wal
uuid: "38482fb2b4114b81a9693eccc95c7f31"
format_stamp: "Formatted at 2026-08-11 06:30:56 on dist-test-slave-gkw7"
server_key: "a9fd85a73b131e6857c4f8d5e7217ba1"
server_key_iv: "d005d2eab4206d4aa21280ba3b31d4a2"
server_key_version: "encryptionkey@0"
I20260811 06:30:56.061771 12017 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:30:56.085505 12017 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:30:56.086649 12017 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:30:56.087059 12017 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:30:56.088611 12017 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:30:56.090358 12017 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:30:56.090504 12017 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:56.090646 12017 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:30:56.090744 12017 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:56.128230 12017 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.193:43805
I20260811 06:30:56.128295 12222 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.193:43805 every 8 connection(s)
I20260811 06:30:56.129876 12017 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
I20260811 06:30:56.135506 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 12017
I20260811 06:30:56.135725 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-0/wal/instance
I20260811 06:30:56.136574 29631 external_mini_cluster.cc:1428] Setting key 83d7af8d113934427deed2ffcd0b518b
I20260811 06:30:56.139771 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.194:0
--local_ip_for_outbound_sockets=127.28.239.194
--webserver_interface=127.28.239.194
--webserver_port=0
--tserver_master_addrs=127.28.239.254:40043
--builtin_ntp_servers=127.28.239.212:46267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260811 06:30:56.146708 12225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:56.132845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.193:51261 (local address 127.28.239.254:40043)
0811 06:30:56.133322 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:56.133327 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:56.134741 (+ 1414us) server_negotiation.cc:407] Connection header received
0811 06:30:56.137429 (+ 2688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:56.137437 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:56.137589 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:56.137703 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:56.139161 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:56.139965 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:56.142735 (+ 2770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:56.143772 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:56.143951 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:56.145844 (+ 1893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:56.145849 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:56.145853 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:30:56.145875 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:56.145918 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:30:56.145932 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:56.146014 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:56.146299 (+ 285us) server_negotiation.cc:299] Negotiation successful
0811 06:30:56.146393 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":245,"threads_started":1}
I20260811 06:30:56.148291 12226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:56.133590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:40043 (local address 127.28.239.193:51261)
0811 06:30:56.134572 (+ 982us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:56.134610 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:56.137052 (+ 2442us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:56.137995 (+ 943us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:56.138004 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:56.138111 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:56.138928 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:56.138958 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:56.140143 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:56.140151 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:56.142506 (+ 2355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:56.142526 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:56.144115 (+ 1589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:56.144127 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:56.144518 (+ 391us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:56.145400 (+ 882us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 06:30:56.145406 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:56.145467 (+ 61us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 06:30:56.145623 (+ 156us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:30:56.146039 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:30:56.146052 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:30:56.146063 (+ 11us) client_negotiation.cc:769] Sending connection context
0811 06:30:56.146265 (+ 202us) client_negotiation.cc:240] Negotiation successful
0811 06:30:56.146745 (+ 480us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":290,"threads_started":1}
I20260811 06:30:56.150100 12223 heartbeater.cc:347] Connected to a master server at 127.28.239.254:40043
I20260811 06:30:56.150456 12223 heartbeater.cc:464] Registering TS with master...
I20260811 06:30:56.151340 12223 heartbeater.cc:511] Master 127.28.239.254:40043 requested a full tablet report, sending...
I20260811 06:30:56.153806 11921 ts_manager.cc:194] Registered new tserver with Master: 38482fb2b4114b81a9693eccc95c7f31 (127.28.239.193:43805)
I20260811 06:30:56.156162 11921 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.28.239.193:51261
W20260811 06:30:56.493145 12229 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:30:56.493449 12229 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:30:56.493580 12229 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:30:56.500916 12229 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:30:56.501034 12229 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:30:56.501147 12229 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.194
I20260811 06:30:56.510680 12229 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:46267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.239.194
--webserver_port=0
--tserver_master_addrs=127.28.239.254:40043
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.194
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:30:56.512014 12229 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:30:56.513933 12229 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:30:56.528126 12235 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:30:56.528260 12241 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:30:56.528098 12237 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:30:56.529102 12229 server_base.cc:1034] running on GCE node
I20260811 06:30:56.529932 12229 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:30:56.531080 12229 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:30:56.532335 12229 hybrid_clock.cc:648] HybridClock initialized: now 1786429856532288 us; error 52 us; skew 500 ppm
I20260811 06:30:56.532701 12229 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:30:56.535285 12229 webserver.cc:460] Webserver started at http://127.28.239.194:32925/ using document root <none> and password file <none>
I20260811 06:30:56.536060 12229 fs_manager.cc:359] Metadata directory not provided
I20260811 06:30:56.536199 12229 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:30:56.536549 12229 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:30:56.539602 12229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-1/data/instance:
uuid: "5ada0e7710e6407e8471ceed909411b7"
format_stamp: "Formatted at 2026-08-11 06:30:56 on dist-test-slave-gkw7"
server_key: "1b0d227b27432d8217a2a41ed4cb0f4c"
server_key_iv: "699d179435620bfeaa1cf84123c2ccf0"
server_key_version: "encryptionkey@0"
I20260811 06:30:56.540664 12229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-1/wal/instance:
uuid: "5ada0e7710e6407e8471ceed909411b7"
format_stamp: "Formatted at 2026-08-11 06:30:56 on dist-test-slave-gkw7"
server_key: "1b0d227b27432d8217a2a41ed4cb0f4c"
server_key_iv: "699d179435620bfeaa1cf84123c2ccf0"
server_key_version: "encryptionkey@0"
I20260811 06:30:56.547657 12229 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.002s
I20260811 06:30:56.552381 12248 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:56.554139 12229 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.001s
I20260811 06:30:56.554334 12229 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-1/wal
uuid: "5ada0e7710e6407e8471ceed909411b7"
format_stamp: "Formatted at 2026-08-11 06:30:56 on dist-test-slave-gkw7"
server_key: "1b0d227b27432d8217a2a41ed4cb0f4c"
server_key_iv: "699d179435620bfeaa1cf84123c2ccf0"
server_key_version: "encryptionkey@0"
I20260811 06:30:56.554564 12229 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:30:56.585201 12229 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:30:56.586118 12229 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:30:56.586526 12229 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:30:56.587647 12229 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:30:56.589185 12229 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:30:56.589287 12229 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:56.589447 12229 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:30:56.589560 12229 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:56.627252 12229 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.194:34229
I20260811 06:30:56.627364 12426 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.194:34229 every 8 connection(s)
I20260811 06:30:56.628599 12229 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
I20260811 06:30:56.634002 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 12229
I20260811 06:30:56.634253 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-1/wal/instance
I20260811 06:30:56.634847 29631 external_mini_cluster.cc:1428] Setting key 312708510d6907a83d888e34fee12566
I20260811 06:30:56.638201 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.195:0
--local_ip_for_outbound_sockets=127.28.239.195
--webserver_interface=127.28.239.195
--webserver_port=0
--tserver_master_addrs=127.28.239.254:40043
--builtin_ntp_servers=127.28.239.212:46267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260811 06:30:56.644532 12225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:56.631926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.194:59019 (local address 127.28.239.254:40043)
0811 06:30:56.632138 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:56.632143 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:56.633751 (+ 1608us) server_negotiation.cc:407] Connection header received
0811 06:30:56.636231 (+ 2480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:56.636236 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:56.636318 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:56.636408 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:56.637818 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:56.638622 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:56.641170 (+ 2548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:56.641617 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:56.641712 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:56.643762 (+ 2050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:56.643767 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:56.643770 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:30:56.643788 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:56.643821 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:30:56.643832 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:56.643902 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:56.644221 (+ 319us) server_negotiation.cc:299] Negotiation successful
0811 06:30:56.644304 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 06:30:56.646528 12430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:56.632511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:40043 (local address 127.28.239.194:59019)
0811 06:30:56.633492 (+ 981us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:56.633599 (+ 107us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:56.635891 (+ 2292us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:56.636701 (+ 810us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:56.636712 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:56.636850 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:56.637639 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:56.637666 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:56.638767 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:56.638775 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:56.641012 (+ 2237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:56.641031 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:56.641915 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:56.641928 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:56.642417 (+ 489us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:56.643320 (+ 903us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 06:30:56.643327 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:56.643414 (+ 87us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 06:30:56.643541 (+ 127us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:30:56.643944 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:30:56.643959 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:30:56.643972 (+ 13us) client_negotiation.cc:769] Sending connection context
0811 06:30:56.644231 (+ 259us) client_negotiation.cc:240] Negotiation successful
0811 06:30:56.644746 (+ 515us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":307,"threads_started":1}
I20260811 06:30:56.648161 12428 heartbeater.cc:347] Connected to a master server at 127.28.239.254:40043
I20260811 06:30:56.648581 12428 heartbeater.cc:464] Registering TS with master...
I20260811 06:30:56.649812 12428 heartbeater.cc:511] Master 127.28.239.254:40043 requested a full tablet report, sending...
I20260811 06:30:56.651860 11921 ts_manager.cc:194] Registered new tserver with Master: 5ada0e7710e6407e8471ceed909411b7 (127.28.239.194:34229)
I20260811 06:30:56.653146 11921 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.28.239.194:59019
W20260811 06:30:56.992591 12432 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:30:56.992884 12432 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:30:56.992970 12432 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:30:57.000303 12432 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:30:57.000406 12432 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:30:57.000511 12432 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.195
I20260811 06:30:57.009560 12432 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:46267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.239.195
--webserver_port=0
--tserver_master_addrs=127.28.239.254:40043
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.195
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:30:57.010968 12432 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:30:57.012775 12432 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:30:57.026671 12442 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:30:57.026692 12437 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:30:57.026641 12438 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:30:57.027540 12432 server_base.cc:1034] running on GCE node
I20260811 06:30:57.028236 12432 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:30:57.029289 12432 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:30:57.030510 12432 hybrid_clock.cc:648] HybridClock initialized: now 1786429857030424 us; error 80 us; skew 500 ppm
I20260811 06:30:57.030864 12432 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:30:57.033322 12432 webserver.cc:460] Webserver started at http://127.28.239.195:41387/ using document root <none> and password file <none>
I20260811 06:30:57.034144 12432 fs_manager.cc:359] Metadata directory not provided
I20260811 06:30:57.034283 12432 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:30:57.034641 12432 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:30:57.037703 12432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-2/data/instance:
uuid: "e42ef6ad206b4a2a960a19cfacf0d89a"
format_stamp: "Formatted at 2026-08-11 06:30:57 on dist-test-slave-gkw7"
server_key: "e6344e568254478c6ff9c618910b9776"
server_key_iv: "70cda10e006b9f491b92d0a616dca1db"
server_key_version: "encryptionkey@0"
I20260811 06:30:57.038539 12432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-2/wal/instance:
uuid: "e42ef6ad206b4a2a960a19cfacf0d89a"
format_stamp: "Formatted at 2026-08-11 06:30:57 on dist-test-slave-gkw7"
server_key: "e6344e568254478c6ff9c618910b9776"
server_key_iv: "70cda10e006b9f491b92d0a616dca1db"
server_key_version: "encryptionkey@0"
I20260811 06:30:57.043996 12432 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 06:30:57.047799 12450 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:57.049237 12432 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260811 06:30:57.049438 12432 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-2/wal
uuid: "e42ef6ad206b4a2a960a19cfacf0d89a"
format_stamp: "Formatted at 2026-08-11 06:30:57 on dist-test-slave-gkw7"
server_key: "e6344e568254478c6ff9c618910b9776"
server_key_iv: "70cda10e006b9f491b92d0a616dca1db"
server_key_version: "encryptionkey@0"
I20260811 06:30:57.049721 12432 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:30:57.069689 12432 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:30:57.070628 12432 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:30:57.070955 12432 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:30:57.072000 12432 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:30:57.073511 12432 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:30:57.073670 12432 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:57.073813 12432 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:30:57.073905 12432 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:57.108521 12432 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.195:36981
I20260811 06:30:57.108574 12616 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.195:36981 every 8 connection(s)
I20260811 06:30:57.109978 12432 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
I20260811 06:30:57.112154 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 12432
I20260811 06:30:57.112407 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-2/wal/instance
I20260811 06:30:57.113032 29631 external_mini_cluster.cc:1428] Setting key cc1e647ca87e6da645d3ec32bb21bd5c
I20260811 06:30:57.122828 12225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:57.112682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.195:47579 (local address 127.28.239.254:40043)
0811 06:30:57.112911 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:57.112917 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:57.114538 (+ 1621us) server_negotiation.cc:407] Connection header received
0811 06:30:57.116298 (+ 1760us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:57.116303 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:57.116393 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:57.116498 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:57.117612 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:57.118371 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:57.119941 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:57.120357 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:57.120492 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:57.121991 (+ 1499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:57.121996 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:57.121999 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:30:57.122017 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:57.122049 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:30:57.122061 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:57.122159 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:57.122387 (+ 228us) server_negotiation.cc:299] Negotiation successful
0811 06:30:57.122490 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 06:30:57.123910 12621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:57.113256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:40043 (local address 127.28.239.195:47579)
0811 06:30:57.114363 (+ 1107us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:57.114408 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:57.116021 (+ 1613us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:57.116756 (+ 735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:57.116764 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:57.116853 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:57.117422 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:57.117439 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:57.118536 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:57.118540 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:57.119790 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:30:57.119807 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:30:57.120586 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:30:57.120593 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:30:57.120847 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:57.121626 (+ 779us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 06:30:57.121629 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:57.121671 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 06:30:57.121760 (+ 89us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:30:57.122159 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:30:57.122173 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:30:57.122184 (+ 11us) client_negotiation.cc:769] Sending connection context
0811 06:30:57.122374 (+ 190us) client_negotiation.cc:240] Negotiation successful
0811 06:30:57.122797 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":569,"thread_start_us":363,"threads_started":1}
I20260811 06:30:57.125380 12617 heartbeater.cc:347] Connected to a master server at 127.28.239.254:40043
I20260811 06:30:57.125713 12617 heartbeater.cc:464] Registering TS with master...
I20260811 06:30:57.126526 12617 heartbeater.cc:511] Master 127.28.239.254:40043 requested a full tablet report, sending...
I20260811 06:30:57.128089 11921 ts_manager.cc:194] Registered new tserver with Master: e42ef6ad206b4a2a960a19cfacf0d89a (127.28.239.195:36981)
I20260811 06:30:57.129127 11921 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.28.239.195:47579
I20260811 06:30:57.129752 29631 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 06:30:57.136410 12225 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:30:57.134765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58310 (local address 127.28.239.254:40043)
0811 06:30:57.134962 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:57.134967 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:57.135467 (+ 500us) server_negotiation.cc:407] Connection header received
0811 06:30:57.135604 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:57.135607 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:57.135664 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:57.135745 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:57.136133 (+ 388us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58310: BlockingRecv error: recv got EOF from 127.0.0.1:58310 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 06:30:57.137332 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 12017
I20260811 06:30:57.145098 12638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:57.138369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49752 (local address 127.28.239.193:43805)
0811 06:30:57.138926 (+ 557us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:57.138932 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:57.138958 (+ 26us) server_negotiation.cc:407] Connection header received
0811 06:30:57.139009 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:57.139014 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:57.139212 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:57.139333 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:57.139668 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:57.140680 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:57.141958 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:57.142510 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:57.142611 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:57.143645 (+ 1034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:57.143652 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:57.143664 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:30:57.144213 (+ 549us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:57.144256 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:30:57.144276 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:57.144361 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:57.144582 (+ 221us) server_negotiation.cc:299] Negotiation successful
0811 06:30:57.144718 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":288,"threads_started":1}
I20260811 06:30:57.146462 12215 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:30:57.227934 12223 heartbeater.cc:503] Master 127.28.239.254:40043 was elected leader, sending a full tablet report...
I20260811 06:30:57.229334 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 12017
W20260811 06:30:57.254850 11899 connection.cc:535] server connection from 127.28.239.193:51261 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 06:30:57.255173 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 12229
I20260811 06:30:57.263211 12640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:57.256191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44756 (local address 127.28.239.194:34229)
0811 06:30:57.256664 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:57.256671 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:57.256699 (+ 28us) server_negotiation.cc:407] Connection header received
0811 06:30:57.256769 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:57.256776 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:57.257014 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:57.257142 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:57.257631 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:57.258637 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:57.259947 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:57.260625 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:57.260768 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:57.261873 (+ 1105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:57.261881 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:57.261894 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:30:57.262498 (+ 604us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:57.262551 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:30:57.262571 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:57.262675 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:57.262806 (+ 131us) server_negotiation.cc:299] Negotiation successful
0811 06:30:57.262918 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":217,"threads_started":1}
I20260811 06:30:57.264463 12420 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:30:57.354923 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 12229
I20260811 06:30:57.379189 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 12432
I20260811 06:30:57.388221 12645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:57.380270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45938 (local address 127.28.239.195:36981)
0811 06:30:57.380972 (+ 702us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:57.380979 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:57.381011 (+ 32us) server_negotiation.cc:407] Connection header received
0811 06:30:57.381074 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:57.381080 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:57.381321 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:57.381503 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:57.382069 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:57.383188 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:57.384583 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:30:57.385167 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:30:57.385310 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:30:57.386537 (+ 1227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:57.386546 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:57.386562 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:30:57.387290 (+ 728us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:57.387341 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:30:57.387362 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:57.387494 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:57.387671 (+ 177us) server_negotiation.cc:299] Negotiation successful
0811 06:30:57.387831 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":537,"thread_start_us":377,"threads_started":1}
I20260811 06:30:57.389602 12611 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:30:57.472097 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 12432
I20260811 06:30:57.500208 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 11872
I20260811 06:30:57.501757 11970 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:30:57.609628 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 11872
2026-08-11T06:30:57Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2631 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-11T06:30:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:30:57Z Disabled control of system clock
I20260811 06:30:57.710372 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429755927508-29631-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429755927508-29631-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429755927508-29631-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:45503
--webserver_interface=127.28.239.254
--webserver_port=0
--builtin_ntp_servers=127.28.239.212:35609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.239.254:45503
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 06:30:58.073444 12654 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:30:58.073881 12654 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:30:58.074059 12654 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:30:58.081730 12654 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:30:58.081872 12654 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:30:58.081982 12654 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:30:58.082091 12654 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:30:58.082180 12654 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:30:58.091987 12654 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:35609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429755927508-29631-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429755927508-29631-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.239.254:45503
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:45503
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.239.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429755927508-29631-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:30:58.093470 12654 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:30:58.095572 12654 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:30:58.108083 12661 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:30:58.108000 12666 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:30:58.108039 12662 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:30:58.108987 12654 server_base.cc:1034] running on GCE node
I20260811 06:30:58.109822 12654 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:30:58.111251 12654 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:30:58.112465 12654 hybrid_clock.cc:648] HybridClock initialized: now 1786429858112411 us; error 43 us; skew 500 ppm
I20260811 06:30:58.112843 12654 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:30:58.115382 12654 webserver.cc:460] Webserver started at http://127.28.239.254:34279/ using document root <none> and password file <none>
I20260811 06:30:58.116135 12654 fs_manager.cc:359] Metadata directory not provided
I20260811 06:30:58.116264 12654 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:30:58.116628 12654 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:30:58.119740 12654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429755927508-29631-0/minicluster-data/master-0/data/instance:
uuid: "a23b9d6c85e743dc96eddf790ec5aebe"
format_stamp: "Formatted at 2026-08-11 06:30:58 on dist-test-slave-gkw7"
server_key: "e31d10d51d54d2244b948323da8c5014"
server_key_iv: "7abbb01dd9cc885f201517fce8cd06d7"
server_key_version: "encryptionkey@0"
I20260811 06:30:58.120569 12654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429755927508-29631-0/minicluster-data/master-0/wal/instance:
uuid: "a23b9d6c85e743dc96eddf790ec5aebe"
format_stamp: "Formatted at 2026-08-11 06:30:58 on dist-test-slave-gkw7"
server_key: "e31d10d51d54d2244b948323da8c5014"
server_key_iv: "7abbb01dd9cc885f201517fce8cd06d7"
server_key_version: "encryptionkey@0"
I20260811 06:30:58.126230 12654 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 06:30:58.129979 12673 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:58.131376 12654 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 06:30:58.131558 12654 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429755927508-29631-0/minicluster-data/master-0/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429755927508-29631-0/minicluster-data/master-0/wal
uuid: "a23b9d6c85e743dc96eddf790ec5aebe"
format_stamp: "Formatted at 2026-08-11 06:30:58 on dist-test-slave-gkw7"
server_key: "e31d10d51d54d2244b948323da8c5014"
server_key_iv: "7abbb01dd9cc885f201517fce8cd06d7"
server_key_version: "encryptionkey@0"
I20260811 06:30:58.131786 12654 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429755927508-29631-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429755927508-29631-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429755927508-29631-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:30:58.137720 12654 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:30:58.138576 12654 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:30:58.138891 12654 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:30:58.156925 12654 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.254:45503
I20260811 06:30:58.156961 12770 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.254:45503 every 8 connection(s)
I20260811 06:30:58.158373 12654 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
I20260811 06:30:58.160890 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 12654
I20260811 06:30:58.161124 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429755927508-29631-0/minicluster-data/master-0/wal/instance
I20260811 06:30:58.161926 29631 external_mini_cluster.cc:1428] Setting key c9373aff377ef80e61bea909f0a67a3e
I20260811 06:30:58.162616 12772 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:30:58.171272 12776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:58.164035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52532 (local address 127.28.239.254:45503)
0811 06:30:58.165039 (+ 1004us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:58.165051 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:58.165095 (+ 44us) server_negotiation.cc:407] Connection header received
0811 06:30:58.167494 (+ 2399us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:58.167518 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:58.167964 (+ 446us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:58.168404 (+ 440us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:58.169613 (+ 1209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:58.169620 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:58.169638 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:30:58.169656 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:58.169720 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:30:58.169737 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:58.170003 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:58.170145 (+ 142us) server_negotiation.cc:299] Negotiation successful
0811 06:30:58.170395 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":584,"thread_start_us":362,"threads_started":1}
I20260811 06:30:58.176631 12772 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a23b9d6c85e743dc96eddf790ec5aebe: Bootstrap starting.
I20260811 06:30:58.179888 12772 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a23b9d6c85e743dc96eddf790ec5aebe: Neither blocks nor log segments found. Creating new log.
I20260811 06:30:58.181003 12772 log.cc:824] T 00000000000000000000000000000000 P a23b9d6c85e743dc96eddf790ec5aebe: Log is configured to *not* fsync() on all Append() calls
I20260811 06:30:58.184410 12772 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a23b9d6c85e743dc96eddf790ec5aebe: No bootstrap required, opened a new log
I20260811 06:30:58.192631 12772 raft_consensus.cc:348] T 00000000000000000000000000000000 P a23b9d6c85e743dc96eddf790ec5aebe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a23b9d6c85e743dc96eddf790ec5aebe" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 45503 } }
I20260811 06:30:58.193066 12772 raft_consensus.cc:374] T 00000000000000000000000000000000 P a23b9d6c85e743dc96eddf790ec5aebe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:30:58.193202 12772 raft_consensus.cc:729] T 00000000000000000000000000000000 P a23b9d6c85e743dc96eddf790ec5aebe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a23b9d6c85e743dc96eddf790ec5aebe, State: Initialized, Role: FOLLOWER
I20260811 06:30:58.193877 12772 consensus_queue.cc:260] T 00000000000000000000000000000000 P a23b9d6c85e743dc96eddf790ec5aebe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a23b9d6c85e743dc96eddf790ec5aebe" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 45503 } }
I20260811 06:30:58.194139 12772 raft_consensus.cc:388] T 00000000000000000000000000000000 P a23b9d6c85e743dc96eddf790ec5aebe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:30:58.194305 12772 raft_consensus.cc:482] T 00000000000000000000000000000000 P a23b9d6c85e743dc96eddf790ec5aebe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:30:58.194499 12772 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a23b9d6c85e743dc96eddf790ec5aebe [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:30:58.197171 12772 raft_consensus.cc:504] T 00000000000000000000000000000000 P a23b9d6c85e743dc96eddf790ec5aebe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a23b9d6c85e743dc96eddf790ec5aebe" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 45503 } }
I20260811 06:30:58.197850 12772 leader_election.cc:302] T 00000000000000000000000000000000 P a23b9d6c85e743dc96eddf790ec5aebe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a23b9d6c85e743dc96eddf790ec5aebe; no voters:
I20260811 06:30:58.198506 12772 leader_election.cc:288] T 00000000000000000000000000000000 P a23b9d6c85e743dc96eddf790ec5aebe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:30:58.198669 12778 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a23b9d6c85e743dc96eddf790ec5aebe [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:30:58.199430 12778 raft_consensus.cc:686] T 00000000000000000000000000000000 P a23b9d6c85e743dc96eddf790ec5aebe [term 1 LEADER]: Becoming Leader. State: Replica: a23b9d6c85e743dc96eddf790ec5aebe, State: Running, Role: LEADER
I20260811 06:30:58.200099 12778 consensus_queue.cc:237] T 00000000000000000000000000000000 P a23b9d6c85e743dc96eddf790ec5aebe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a23b9d6c85e743dc96eddf790ec5aebe" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 45503 } }
I20260811 06:30:58.200757 12772 sys_catalog.cc:564] T 00000000000000000000000000000000 P a23b9d6c85e743dc96eddf790ec5aebe [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:30:58.203426 12780 sys_catalog.cc:455] T 00000000000000000000000000000000 P a23b9d6c85e743dc96eddf790ec5aebe [sys.catalog]: SysCatalogTable state changed. Reason: New leader a23b9d6c85e743dc96eddf790ec5aebe. Latest consensus state: current_term: 1 leader_uuid: "a23b9d6c85e743dc96eddf790ec5aebe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a23b9d6c85e743dc96eddf790ec5aebe" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 45503 } } }
I20260811 06:30:58.203976 12780 sys_catalog.cc:458] T 00000000000000000000000000000000 P a23b9d6c85e743dc96eddf790ec5aebe [sys.catalog]: This master's current role is: LEADER
I20260811 06:30:58.203449 12779 sys_catalog.cc:455] T 00000000000000000000000000000000 P a23b9d6c85e743dc96eddf790ec5aebe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a23b9d6c85e743dc96eddf790ec5aebe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a23b9d6c85e743dc96eddf790ec5aebe" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 45503 } } }
I20260811 06:30:58.204128 12779 sys_catalog.cc:458] T 00000000000000000000000000000000 P a23b9d6c85e743dc96eddf790ec5aebe [sys.catalog]: This master's current role is: LEADER
I20260811 06:30:58.205999 12789 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:30:58.210767 12789 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:30:58.219462 12789 catalog_manager.cc:1347] Generated new cluster ID: 00bba600b68a4d1488a9b279fa4baa8c
I20260811 06:30:58.219594 12789 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:30:58.249455 12789 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:30:58.250025 12789 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:30:58.264285 12789 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a23b9d6c85e743dc96eddf790ec5aebe: Generated new TSK 0
I20260811 06:30:58.265202 12789 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:30:58.283730 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.193:0
--local_ip_for_outbound_sockets=127.28.239.193
--webserver_interface=127.28.239.193
--webserver_port=0
--tserver_master_addrs=127.28.239.254:45503
--builtin_ntp_servers=127.28.239.212:35609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 06:30:58.661818 12804 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:30:58.662119 12804 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:30:58.662214 12804 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:30:58.669493 12804 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:30:58.669674 12804 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:30:58.669884 12804 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.193
I20260811 06:30:58.679345 12804 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:35609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.239.193
--webserver_port=0
--tserver_master_addrs=127.28.239.254:45503
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.193
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:30:58.680727 12804 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:30:58.682790 12804 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:30:58.696590 12816 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:30:58.696797 12813 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:30:58.696560 12814 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:30:58.697479 12804 server_base.cc:1034] running on GCE node
I20260811 06:30:58.698122 12804 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:30:58.699047 12804 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:30:58.700263 12804 hybrid_clock.cc:648] HybridClock initialized: now 1786429858700217 us; error 35 us; skew 500 ppm
I20260811 06:30:58.700547 12804 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:30:58.707616 12804 webserver.cc:460] Webserver started at http://127.28.239.193:44103/ using document root <none> and password file <none>
I20260811 06:30:58.708359 12804 fs_manager.cc:359] Metadata directory not provided
I20260811 06:30:58.708449 12804 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:30:58.708720 12804 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:30:58.711663 12804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-0/data/instance:
uuid: "e02399b7179947b1b154db6ae2ac450e"
format_stamp: "Formatted at 2026-08-11 06:30:58 on dist-test-slave-gkw7"
server_key: "b4f307b1025b69f80387efb6c0822da7"
server_key_iv: "f53bfe6be26bbd013e418e75e641889b"
server_key_version: "encryptionkey@0"
I20260811 06:30:58.712427 12804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-0/wal/instance:
uuid: "e02399b7179947b1b154db6ae2ac450e"
format_stamp: "Formatted at 2026-08-11 06:30:58 on dist-test-slave-gkw7"
server_key: "b4f307b1025b69f80387efb6c0822da7"
server_key_iv: "f53bfe6be26bbd013e418e75e641889b"
server_key_version: "encryptionkey@0"
I20260811 06:30:58.717660 12804 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.006s
I20260811 06:30:58.721071 12828 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:58.722709 12804 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:30:58.722864 12804 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-0/wal
uuid: "e02399b7179947b1b154db6ae2ac450e"
format_stamp: "Formatted at 2026-08-11 06:30:58 on dist-test-slave-gkw7"
server_key: "b4f307b1025b69f80387efb6c0822da7"
server_key_iv: "f53bfe6be26bbd013e418e75e641889b"
server_key_version: "encryptionkey@0"
I20260811 06:30:58.723114 12804 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:30:58.729247 12804 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:30:58.730207 12804 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:30:58.730453 12804 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:30:58.731400 12804 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:30:58.732848 12804 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:30:58.732939 12804 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:58.733026 12804 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:30:58.733090 12804 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:58.769361 12804 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.193:46599
I20260811 06:30:58.769459 12993 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.193:46599 every 8 connection(s)
I20260811 06:30:58.770833 12804 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
I20260811 06:30:58.779654 13000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:58.773794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.193:46697 (local address 127.28.239.254:45503)
0811 06:30:58.774325 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:58.774329 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:58.775435 (+ 1106us) server_negotiation.cc:407] Connection header received
0811 06:30:58.777190 (+ 1755us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:58.777195 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:58.777281 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:58.777404 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:58.778807 (+ 1403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:58.778813 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:58.778819 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:30:58.778842 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:58.778886 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:30:58.778898 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:58.779036 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:58.779234 (+ 198us) server_negotiation.cc:299] Negotiation successful
0811 06:30:58.779319 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":288,"threads_started":1}
I20260811 06:30:58.779811 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 12804
I20260811 06:30:58.780041 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-0/wal/instance
I20260811 06:30:58.780686 29631 external_mini_cluster.cc:1428] Setting key 9ed92d9b287143d229adc59ceaa8078d
I20260811 06:30:58.780686 13001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:58.774312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:45503 (local address 127.28.239.193:46697)
0811 06:30:58.775258 (+ 946us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:58.775295 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:58.776906 (+ 1611us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:58.777677 (+ 771us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:58.777685 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:58.777810 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:58.778506 (+ 696us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 06:30:58.778509 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:58.778547 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 06:30:58.778632 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:30:58.779000 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:30:58.779013 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:30:58.779021 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 06:30:58.779323 (+ 302us) client_negotiation.cc:240] Negotiation successful
0811 06:30:58.779595 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":281,"threads_started":1}
I20260811 06:30:58.782402 12997 heartbeater.cc:347] Connected to a master server at 127.28.239.254:45503
I20260811 06:30:58.782749 12997 heartbeater.cc:464] Registering TS with master...
I20260811 06:30:58.783424 12997 heartbeater.cc:511] Master 127.28.239.254:45503 requested a full tablet report, sending...
I20260811 06:30:58.783962 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.194:0
--local_ip_for_outbound_sockets=127.28.239.194
--webserver_interface=127.28.239.194
--webserver_port=0
--tserver_master_addrs=127.28.239.254:45503
--builtin_ntp_servers=127.28.239.212:35609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260811 06:30:58.785619 12695 ts_manager.cc:194] Registered new tserver with Master: e02399b7179947b1b154db6ae2ac450e (127.28.239.193:46599)
W20260811 06:30:59.139556 13004 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:30:59.139863 13004 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:30:59.140010 13004 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:30:59.147583 13004 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:30:59.147694 13004 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:30:59.147872 13004 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.194
I20260811 06:30:59.157299 13004 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:35609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.239.194
--webserver_port=0
--tserver_master_addrs=127.28.239.254:45503
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.194
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:30:59.158682 13004 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:30:59.160537 13004 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:30:59.177980 13012 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:30:59.178086 13016 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:30:59.178114 13011 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:30:59.178853 13004 server_base.cc:1034] running on GCE node
I20260811 06:30:59.179553 13004 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:30:59.180728 13004 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:30:59.181979 13004 hybrid_clock.cc:648] HybridClock initialized: now 1786429859181916 us; error 60 us; skew 500 ppm
I20260811 06:30:59.182382 13004 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:30:59.185384 13004 webserver.cc:460] Webserver started at http://127.28.239.194:38551/ using document root <none> and password file <none>
I20260811 06:30:59.186259 13004 fs_manager.cc:359] Metadata directory not provided
I20260811 06:30:59.186408 13004 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:30:59.186789 13004 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:30:59.190130 13004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-1/data/instance:
uuid: "67b8e0d4342b492699f4f10415be35d0"
format_stamp: "Formatted at 2026-08-11 06:30:59 on dist-test-slave-gkw7"
server_key: "035db0c85e4688c7b12b4a82dcfab54b"
server_key_iv: "c7810f94d579ca7645c49fc35367326a"
server_key_version: "encryptionkey@0"
I20260811 06:30:59.191038 13004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-1/wal/instance:
uuid: "67b8e0d4342b492699f4f10415be35d0"
format_stamp: "Formatted at 2026-08-11 06:30:59 on dist-test-slave-gkw7"
server_key: "035db0c85e4688c7b12b4a82dcfab54b"
server_key_iv: "c7810f94d579ca7645c49fc35367326a"
server_key_version: "encryptionkey@0"
I20260811 06:30:59.197033 13004 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260811 06:30:59.201296 13025 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:59.202965 13004 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260811 06:30:59.203163 13004 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-1/wal
uuid: "67b8e0d4342b492699f4f10415be35d0"
format_stamp: "Formatted at 2026-08-11 06:30:59 on dist-test-slave-gkw7"
server_key: "035db0c85e4688c7b12b4a82dcfab54b"
server_key_iv: "c7810f94d579ca7645c49fc35367326a"
server_key_version: "encryptionkey@0"
I20260811 06:30:59.203423 13004 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:30:59.209914 13004 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:30:59.210916 13004 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:30:59.211198 13004 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:30:59.212231 13004 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:30:59.213918 13004 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:30:59.214046 13004 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:59.214174 13004 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:30:59.214263 13004 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:59.248639 13004 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.194:36453
I20260811 06:30:59.248693 13193 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.194:36453 every 8 connection(s)
I20260811 06:30:59.250098 13004 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
I20260811 06:30:59.255631 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 13004
I20260811 06:30:59.255841 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-1/wal/instance
I20260811 06:30:59.256409 29631 external_mini_cluster.cc:1428] Setting key 29779ae2746ca2ed9b0160a8f6d09f61
I20260811 06:30:59.259052 13000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:59.252756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.194:53941 (local address 127.28.239.254:45503)
0811 06:30:59.252984 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:59.252991 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:59.254421 (+ 1430us) server_negotiation.cc:407] Connection header received
0811 06:30:59.256594 (+ 2173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:59.256598 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:59.256656 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:59.256747 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:59.258289 (+ 1542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:59.258298 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:59.258303 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:30:59.258322 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:59.258365 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:30:59.258378 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:59.258516 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:59.258705 (+ 189us) server_negotiation.cc:299] Negotiation successful
0811 06:30:59.258784 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 06:30:59.259486 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.195:0
--local_ip_for_outbound_sockets=127.28.239.195
--webserver_interface=127.28.239.195
--webserver_port=0
--tserver_master_addrs=127.28.239.254:45503
--builtin_ntp_servers=127.28.239.212:35609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260811 06:30:59.260090 13198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:59.253315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:45503 (local address 127.28.239.194:53941)
0811 06:30:59.254233 (+ 918us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:59.254295 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:59.256230 (+ 1935us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:59.257028 (+ 798us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:59.257042 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:59.257154 (+ 112us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:59.257988 (+ 834us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 06:30:59.257991 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:59.258029 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 06:30:59.258115 (+ 86us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:30:59.258474 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:30:59.258487 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:30:59.258497 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 06:30:59.258796 (+ 299us) client_negotiation.cc:240] Negotiation successful
0811 06:30:59.259063 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":288,"threads_started":1}
I20260811 06:30:59.261595 13195 heartbeater.cc:347] Connected to a master server at 127.28.239.254:45503
I20260811 06:30:59.261910 13195 heartbeater.cc:464] Registering TS with master...
I20260811 06:30:59.262509 13195 heartbeater.cc:511] Master 127.28.239.254:45503 requested a full tablet report, sending...
I20260811 06:30:59.264093 12695 ts_manager.cc:194] Registered new tserver with Master: 67b8e0d4342b492699f4f10415be35d0 (127.28.239.194:36453)
W20260811 06:30:59.617015 13199 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:30:59.617370 13199 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:30:59.617583 13199 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:30:59.626670 13199 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:30:59.626827 13199 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:30:59.627013 13199 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.195
I20260811 06:30:59.636756 13199 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:35609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.239.195
--webserver_port=0
--tserver_master_addrs=127.28.239.254:45503
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.195
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:30:59.638348 13199 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:30:59.640236 13199 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:30:59.654618 13209 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:30:59.654561 13211 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:30:59.654659 13207 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:30:59.655328 13199 server_base.cc:1034] running on GCE node
I20260811 06:30:59.656154 13199 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:30:59.657265 13199 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:30:59.658573 13199 hybrid_clock.cc:648] HybridClock initialized: now 1786429859658481 us; error 83 us; skew 500 ppm
I20260811 06:30:59.658995 13199 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:30:59.664247 13199 webserver.cc:460] Webserver started at http://127.28.239.195:32943/ using document root <none> and password file <none>
I20260811 06:30:59.665102 13199 fs_manager.cc:359] Metadata directory not provided
I20260811 06:30:59.665259 13199 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:30:59.665720 13199 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:30:59.669031 13199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-2/data/instance:
uuid: "82a22259a2874efc938e1bcfe64c2e1b"
format_stamp: "Formatted at 2026-08-11 06:30:59 on dist-test-slave-gkw7"
server_key: "2f1b98976a8a843bb9ebf3859f33cfc9"
server_key_iv: "1ee458783cd2c69f44ec06d6d181d5e9"
server_key_version: "encryptionkey@0"
I20260811 06:30:59.670023 13199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-2/wal/instance:
uuid: "82a22259a2874efc938e1bcfe64c2e1b"
format_stamp: "Formatted at 2026-08-11 06:30:59 on dist-test-slave-gkw7"
server_key: "2f1b98976a8a843bb9ebf3859f33cfc9"
server_key_iv: "1ee458783cd2c69f44ec06d6d181d5e9"
server_key_version: "encryptionkey@0"
I20260811 06:30:59.675921 13199 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260811 06:30:59.679821 13224 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:59.681326 13199 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 06:30:59.681633 13199 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-2/wal
uuid: "82a22259a2874efc938e1bcfe64c2e1b"
format_stamp: "Formatted at 2026-08-11 06:30:59 on dist-test-slave-gkw7"
server_key: "2f1b98976a8a843bb9ebf3859f33cfc9"
server_key_iv: "1ee458783cd2c69f44ec06d6d181d5e9"
server_key_version: "encryptionkey@0"
I20260811 06:30:59.681953 13199 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:30:59.688639 13199 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:30:59.689522 13199 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:30:59.689863 13199 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:30:59.690862 13199 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:30:59.692442 13199 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:30:59.692593 13199 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:59.692742 13199 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:30:59.692843 13199 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:30:59.729879 13199 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.195:43839
I20260811 06:30:59.729980 13384 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.195:43839 every 8 connection(s)
I20260811 06:30:59.731380 13199 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
I20260811 06:30:59.734375 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 13199
I20260811 06:30:59.734580 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786429755927508-29631-0/minicluster-data/ts-2/wal/instance
I20260811 06:30:59.735040 29631 external_mini_cluster.cc:1428] Setting key 0531b2bd40a0ae1193c1d9afb519e5e3
I20260811 06:30:59.740752 13000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:59.734429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.195:57719 (local address 127.28.239.254:45503)
0811 06:30:59.734665 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:59.734671 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:59.736528 (+ 1857us) server_negotiation.cc:407] Connection header received
0811 06:30:59.738416 (+ 1888us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:59.738421 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:59.738494 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:59.738607 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:59.739944 (+ 1337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:59.739950 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:59.739955 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:30:59.739974 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:59.740017 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:30:59.740028 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:59.740156 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:59.740396 (+ 240us) server_negotiation.cc:299] Negotiation successful
0811 06:30:59.740466 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260811 06:30:59.741765 13391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:59.735053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:45503 (local address 127.28.239.195:57719)
0811 06:30:59.736315 (+ 1262us) negotiation.cc:107] Waiting for socket to connect
0811 06:30:59.736368 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 06:30:59.738139 (+ 1771us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:30:59.738855 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:30:59.738863 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:30:59.738957 (+ 94us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:30:59.739644 (+ 687us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 06:30:59.739646 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:30:59.739685 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 06:30:59.739772 (+ 87us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:30:59.740178 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:30:59.740196 (+ 18us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:30:59.740203 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 06:30:59.740482 (+ 279us) client_negotiation.cc:240] Negotiation successful
0811 06:30:59.740745 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":628,"thread_start_us":403,"threads_started":1}
I20260811 06:30:59.743067 13388 heartbeater.cc:347] Connected to a master server at 127.28.239.254:45503
I20260811 06:30:59.743367 13388 heartbeater.cc:464] Registering TS with master...
I20260811 06:30:59.743963 13388 heartbeater.cc:511] Master 127.28.239.254:45503 requested a full tablet report, sending...
I20260811 06:30:59.745348 12695 ts_manager.cc:194] Registered new tserver with Master: 82a22259a2874efc938e1bcfe64c2e1b (127.28.239.195:43839)
I20260811 06:30:59.750407 29631 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 06:30:59.756968 13000 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:30:59.755218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52538 (local address 127.28.239.254:45503)
0811 06:30:59.755466 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:59.755476 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:59.756029 (+ 553us) server_negotiation.cc:407] Connection header received
0811 06:30:59.756151 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:59.756155 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:59.756215 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:59.756312 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:59.756694 (+ 382us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52538: BlockingRecv error: recv got EOF from 127.0.0.1:52538 (error 108)
Metrics: {"server-negotiator.queue_time_us":98}
I20260811 06:30:59.757707 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 12804
I20260811 06:30:59.761883 13399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:59.758775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42700 (local address 127.28.239.193:46599)
0811 06:30:59.759338 (+ 563us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:59.759353 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:59.759385 (+ 32us) server_negotiation.cc:407] Connection header received
0811 06:30:59.759455 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:59.759461 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:59.759693 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:59.759878 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:59.760992 (+ 1114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:59.760999 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:59.761011 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:30:59.761030 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:59.761075 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:30:59.761090 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:59.761261 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:59.761417 (+ 156us) server_negotiation.cc:299] Negotiation successful
0811 06:30:59.761564 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":432,"thread_start_us":273,"threads_started":1}
I20260811 06:30:59.763186 12988 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:30:59.867064 12997 heartbeater.cc:503] Master 127.28.239.254:45503 was elected leader, sending a full tablet report...
I20260811 06:30:59.868568 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 12804
W20260811 06:30:59.893785 12681 connection.cc:535] server connection from 127.28.239.193:46697 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 06:30:59.894136 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 13004
I20260811 06:30:59.898087 13403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:30:59.895224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51818 (local address 127.28.239.194:36453)
0811 06:30:59.895815 (+ 591us) server_negotiation.cc:206] Beginning negotiation
0811 06:30:59.895822 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:30:59.895851 (+ 29us) server_negotiation.cc:407] Connection header received
0811 06:30:59.895918 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:30:59.895925 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:30:59.896159 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:30:59.896303 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:30:59.897280 (+ 977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:30:59.897287 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:30:59.897300 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:30:59.897318 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:30:59.897360 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:30:59.897375 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:30:59.897487 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 06:30:59.897640 (+ 153us) server_negotiation.cc:299] Negotiation successful
0811 06:30:59.897760 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":436,"thread_start_us":263,"threads_started":1}
I20260811 06:30:59.899762 13189 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:30:59.989698 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 13004
I20260811 06:31:00.019171 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 13199
I20260811 06:31:00.023365 13405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:31:00.020274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49308 (local address 127.28.239.195:43839)
0811 06:31:00.020969 (+ 695us) server_negotiation.cc:206] Beginning negotiation
0811 06:31:00.020977 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:31:00.021010 (+ 33us) server_negotiation.cc:407] Connection header received
0811 06:31:00.021076 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:31:00.021082 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:31:00.021320 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:31:00.021452 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:31:00.022602 (+ 1150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:31:00.022609 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:31:00.022621 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:31:00.022638 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:31:00.022677 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:31:00.022690 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:31:00.022829 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 06:31:00.022881 (+ 52us) server_negotiation.cc:299] Negotiation successful
0811 06:31:00.022998 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":539,"thread_start_us":359,"threads_started":1}
I20260811 06:31:00.024681 13374 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:31:00.107998 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 13199
I20260811 06:31:00.129208 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 12654
I20260811 06:31:00.130561 12762 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:31:00.234615 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 12654
2026-08-11T06:31:00Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2623 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:31:00 dist-test-slave-gkw7 krb5kdc[13415](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:31:00 dist-test-slave-gkw7 krb5kdc[13415](info): set up 2 sockets
Aug 11 06:31:00 dist-test-slave-gkw7 krb5kdc[13415](info): commencing operation
krb5kdc: starting...
W20260811 06:31:01.594543 29631 mini_kdc.cc:121] Time spent starting KDC: real 1.307s user 0.006s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/krb5kdc/test-user.keytab.
Aug 11 06:31:01 dist-test-slave-gkw7 krb5kdc[13415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429861, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:31:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:31:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.239.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.239.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.239.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab.
Entry for principal kudu/127.28.239.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab.
I20260811 06:31:01.903266 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:42313
--webserver_interface=127.28.239.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab
--principal=kudu/127.28.239.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:45063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.239.254:42313
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:31:02.272328 13434 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:31:02.272778 13434 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:31:02.272992 13434 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:31:02.280390 13434 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:31:02.280516 13434 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:31:02.280616 13434 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:31:02.280709 13434 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:31:02.280774 13434 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:31:02.291273 13434 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:45063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.239.254:42313
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab
--principal=kudu/127.28.239.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:42313
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.239.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:31:02.292768 13434 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:31:02.294212 13434 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:31:02.307025 13440 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:31:02.307044 13441 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:31:02.307034 13443 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:31:02.308039 13434 server_base.cc:1034] running on GCE node
I20260811 06:31:02.308923 13434 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:31:02.310446 13434 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:31:02.311664 13434 hybrid_clock.cc:648] HybridClock initialized: now 1786429862311612 us; error 43 us; skew 500 ppm
I20260811 06:31:02.312037 13434 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:31:02 dist-test-slave-gkw7 krb5kdc[13415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429862, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:31:02.315487 13434 init.cc:377] Logged in from keytab as kudu/127.28.239.254@KRBTEST.COM (short username kudu)
I20260811 06:31:02.317211 13434 webserver.cc:460] Webserver started at http://127.28.239.254:36987/ using document root <none> and password file <none>
I20260811 06:31:02.317998 13434 fs_manager.cc:359] Metadata directory not provided
I20260811 06:31:02.318156 13434 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:31:02.318495 13434 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:31:02.321743 13434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/minicluster-data/master-0/data/instance:
uuid: "f7db7631b20b4aaf96431f66aa172ecf"
format_stamp: "Formatted at 2026-08-11 06:31:02 on dist-test-slave-gkw7"
server_key: "6ab4d3f250736305cbbbf43d913cdc2a"
server_key_iv: "c1bae1af2983ad85ad6d7bead8b36a38"
server_key_version: "encryptionkey@0"
I20260811 06:31:02.322582 13434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/minicluster-data/master-0/wal/instance:
uuid: "f7db7631b20b4aaf96431f66aa172ecf"
format_stamp: "Formatted at 2026-08-11 06:31:02 on dist-test-slave-gkw7"
server_key: "6ab4d3f250736305cbbbf43d913cdc2a"
server_key_iv: "c1bae1af2983ad85ad6d7bead8b36a38"
server_key_version: "encryptionkey@0"
I20260811 06:31:02.328110 13434 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.002s
I20260811 06:31:02.331880 13457 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:31:02.333374 13434 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 06:31:02.333634 13434 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/minicluster-data/master-0/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/minicluster-data/master-0/wal
uuid: "f7db7631b20b4aaf96431f66aa172ecf"
format_stamp: "Formatted at 2026-08-11 06:31:02 on dist-test-slave-gkw7"
server_key: "6ab4d3f250736305cbbbf43d913cdc2a"
server_key_iv: "c1bae1af2983ad85ad6d7bead8b36a38"
server_key_version: "encryptionkey@0"
I20260811 06:31:02.333850 13434 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:31:02.360630 13434 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:31:02.365397 13434 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:31:02.365767 13434 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:31:02.386078 13434 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.254:42313
I20260811 06:31:02.386081 13538 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.254:42313 every 8 connection(s)
I20260811 06:31:02.387583 13434 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
I20260811 06:31:02.391638 13539 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:31:02.396626 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 13434
I20260811 06:31:02.396895 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/minicluster-data/master-0/wal/instance
I20260811 06:31:02.397883 29631 external_mini_cluster.cc:1428] Setting key 409ef9d87a59492fe191de17bb16f600
I20260811 06:31:02.405639 13539 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7db7631b20b4aaf96431f66aa172ecf: Bootstrap starting.
I20260811 06:31:02.409067 13539 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f7db7631b20b4aaf96431f66aa172ecf: Neither blocks nor log segments found. Creating new log.
I20260811 06:31:02.410372 13539 log.cc:824] T 00000000000000000000000000000000 P f7db7631b20b4aaf96431f66aa172ecf: Log is configured to *not* fsync() on all Append() calls
Aug 11 06:31:02 dist-test-slave-gkw7 krb5kdc[13415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429861, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:31:02.414357 13539 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7db7631b20b4aaf96431f66aa172ecf: No bootstrap required, opened a new log
I20260811 06:31:02.425803 13542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:31:02.400344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48496 (local address 127.28.239.254:42313)
0811 06:31:02.401520 (+ 1176us) server_negotiation.cc:206] Beginning negotiation
0811 06:31:02.401604 (+ 84us) server_negotiation.cc:399] Waiting for connection header
0811 06:31:02.401670 (+ 66us) server_negotiation.cc:407] Connection header received
0811 06:31:02.403756 (+ 2086us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:31:02.403780 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:31:02.404217 (+ 437us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:31:02.404625 (+ 408us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:31:02.406335 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:31:02.407660 (+ 1325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:31:02.408975 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:31:02.409817 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:31:02.410024 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:31:02.415431 (+ 5407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:31:02.415442 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:31:02.415462 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:31:02.415517 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:31:02.420149 (+ 4632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:31:02.421026 (+ 877us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:31:02.421041 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:31:02.421050 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:31:02.421317 (+ 267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:31:02.421869 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:31:02.421874 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:31:02.421880 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:31:02.422725 (+ 845us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:31:02.423074 (+ 349us) server_negotiation.cc:1035] Waiting for connection context
0811 06:31:02.423536 (+ 462us) server_negotiation.cc:299] Negotiation successful
0811 06:31:02.424098 (+ 562us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":956,"thread_start_us":755,"threads_started":1}
I20260811 06:31:02.427130 13539 raft_consensus.cc:348] T 00000000000000000000000000000000 P f7db7631b20b4aaf96431f66aa172ecf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7db7631b20b4aaf96431f66aa172ecf" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 42313 } }
I20260811 06:31:02.427717 13539 raft_consensus.cc:374] T 00000000000000000000000000000000 P f7db7631b20b4aaf96431f66aa172ecf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:31:02.427846 13539 raft_consensus.cc:729] T 00000000000000000000000000000000 P f7db7631b20b4aaf96431f66aa172ecf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7db7631b20b4aaf96431f66aa172ecf, State: Initialized, Role: FOLLOWER
I20260811 06:31:02.428622 13539 consensus_queue.cc:260] T 00000000000000000000000000000000 P f7db7631b20b4aaf96431f66aa172ecf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7db7631b20b4aaf96431f66aa172ecf" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 42313 } }
I20260811 06:31:02.428893 13539 raft_consensus.cc:388] T 00000000000000000000000000000000 P f7db7631b20b4aaf96431f66aa172ecf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:31:02.429100 13539 raft_consensus.cc:482] T 00000000000000000000000000000000 P f7db7631b20b4aaf96431f66aa172ecf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:31:02.429327 13539 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f7db7631b20b4aaf96431f66aa172ecf [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:31:02.431885 13539 raft_consensus.cc:504] T 00000000000000000000000000000000 P f7db7631b20b4aaf96431f66aa172ecf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7db7631b20b4aaf96431f66aa172ecf" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 42313 } }
I20260811 06:31:02.432469 13539 leader_election.cc:302] T 00000000000000000000000000000000 P f7db7631b20b4aaf96431f66aa172ecf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f7db7631b20b4aaf96431f66aa172ecf; no voters:
I20260811 06:31:02.433246 13539 leader_election.cc:288] T 00000000000000000000000000000000 P f7db7631b20b4aaf96431f66aa172ecf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:31:02.433389 13545 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f7db7631b20b4aaf96431f66aa172ecf [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:31:02.434481 13545 raft_consensus.cc:686] T 00000000000000000000000000000000 P f7db7631b20b4aaf96431f66aa172ecf [term 1 LEADER]: Becoming Leader. State: Replica: f7db7631b20b4aaf96431f66aa172ecf, State: Running, Role: LEADER
I20260811 06:31:02.435230 13545 consensus_queue.cc:237] T 00000000000000000000000000000000 P f7db7631b20b4aaf96431f66aa172ecf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7db7631b20b4aaf96431f66aa172ecf" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 42313 } }
I20260811 06:31:02.435992 13539 sys_catalog.cc:564] T 00000000000000000000000000000000 P f7db7631b20b4aaf96431f66aa172ecf [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:31:02.438733 13550 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7db7631b20b4aaf96431f66aa172ecf [sys.catalog]: SysCatalogTable state changed. Reason: New leader f7db7631b20b4aaf96431f66aa172ecf. Latest consensus state: current_term: 1 leader_uuid: "f7db7631b20b4aaf96431f66aa172ecf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7db7631b20b4aaf96431f66aa172ecf" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 42313 } } }
I20260811 06:31:02.439455 13550 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7db7631b20b4aaf96431f66aa172ecf [sys.catalog]: This master's current role is: LEADER
I20260811 06:31:02.439438 13548 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7db7631b20b4aaf96431f66aa172ecf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f7db7631b20b4aaf96431f66aa172ecf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7db7631b20b4aaf96431f66aa172ecf" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 42313 } } }
I20260811 06:31:02.446655 13548 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7db7631b20b4aaf96431f66aa172ecf [sys.catalog]: This master's current role is: LEADER
I20260811 06:31:02.448120 13561 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:31:02.453989 13561 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:31:02.463778 13561 catalog_manager.cc:1347] Generated new cluster ID: 75f34d6d7016402ba888a78ce4f3ae56
I20260811 06:31:02.463929 13561 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:31:02.491184 13561 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:31:02.492661 13561 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:31:02.501348 13561 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f7db7631b20b4aaf96431f66aa172ecf: Generated new TSK 0
I20260811 06:31:02.502295 13561 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.239.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.239.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.239.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.193.keytab.
Entry for principal kudu/127.28.239.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.193.keytab.
I20260811 06:31:02.581311 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.193:0
--local_ip_for_outbound_sockets=127.28.239.193
--webserver_interface=127.28.239.193
--webserver_port=0
--tserver_master_addrs=127.28.239.254:42313
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.193.keytab
--principal=kudu/127.28.239.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:45063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:31:02.946532 13578 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:31:02.946935 13578 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:31:02.947118 13578 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:31:02.954725 13578 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:31:02.954900 13578 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:31:02.955081 13578 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.193
I20260811 06:31:02.965179 13578 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:45063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.193.keytab
--principal=kudu/127.28.239.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.239.193
--webserver_port=0
--tserver_master_addrs=127.28.239.254:42313
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.193
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:31:02.966699 13578 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:31:02.968214 13578 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:31:02.982254 13590 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:31:02.982368 13593 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:31:02.982486 13591 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:31:02.983562 13578 server_base.cc:1034] running on GCE node
I20260811 06:31:02.984205 13578 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:31:02.985268 13578 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:31:02.986531 13578 hybrid_clock.cc:648] HybridClock initialized: now 1786429862986474 us; error 47 us; skew 500 ppm
I20260811 06:31:02.986855 13578 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:31:02 dist-test-slave-gkw7 krb5kdc[13415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429862, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:31:02.990447 13578 init.cc:377] Logged in from keytab as kudu/127.28.239.193@KRBTEST.COM (short username kudu)
I20260811 06:31:02.992054 13578 webserver.cc:460] Webserver started at http://127.28.239.193:43337/ using document root <none> and password file <none>
I20260811 06:31:02.992823 13578 fs_manager.cc:359] Metadata directory not provided
I20260811 06:31:02.992959 13578 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:31:02.993301 13578 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:31:02.996505 13578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/minicluster-data/ts-0/data/instance:
uuid: "330964ae27904a7388a16ff44e34f187"
format_stamp: "Formatted at 2026-08-11 06:31:02 on dist-test-slave-gkw7"
server_key: "b701f420e02a43969e6ac81d395f87d4"
server_key_iv: "a5e1cd9c6536b0984366a2bbb28d81c5"
server_key_version: "encryptionkey@0"
I20260811 06:31:02.997364 13578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/minicluster-data/ts-0/wal/instance:
uuid: "330964ae27904a7388a16ff44e34f187"
format_stamp: "Formatted at 2026-08-11 06:31:02 on dist-test-slave-gkw7"
server_key: "b701f420e02a43969e6ac81d395f87d4"
server_key_iv: "a5e1cd9c6536b0984366a2bbb28d81c5"
server_key_version: "encryptionkey@0"
I20260811 06:31:03.003017 13578 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 06:31:03.007043 13602 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:31:03.008591 13578 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 06:31:03.008803 13578 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/minicluster-data/ts-0/wal
uuid: "330964ae27904a7388a16ff44e34f187"
format_stamp: "Formatted at 2026-08-11 06:31:02 on dist-test-slave-gkw7"
server_key: "b701f420e02a43969e6ac81d395f87d4"
server_key_iv: "a5e1cd9c6536b0984366a2bbb28d81c5"
server_key_version: "encryptionkey@0"
I20260811 06:31:03.009034 13578 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:31:03.022470 13578 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:31:03.027076 13578 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:31:03.027415 13578 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:31:03.028457 13578 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:31:03.030006 13578 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:31:03.030112 13578 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:31:03.030272 13578 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:31:03.030349 13578 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:31:03.075021 13578 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.193:41665
I20260811 06:31:03.075054 13795 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.193:41665 every 8 connection(s)
I20260811 06:31:03.076426 13578 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
I20260811 06:31:03.086779 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 13578
I20260811 06:31:03.087097 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/minicluster-data/ts-0/wal/instance
I20260811 06:31:03.087991 29631 external_mini_cluster.cc:1428] Setting key 9d2bde0aca0069bcb440e2371375adfe
Aug 11 06:31:03 dist-test-slave-gkw7 krb5kdc[13415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429862, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.193@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:31:03.101184 13800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:31:03.079192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.193:37325 (local address 127.28.239.254:42313)
0811 06:31:03.079648 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0811 06:31:03.079653 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:31:03.080812 (+ 1159us) server_negotiation.cc:407] Connection header received
0811 06:31:03.083217 (+ 2405us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:31:03.083222 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:31:03.083313 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:31:03.083427 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:31:03.085063 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:31:03.085906 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:31:03.088349 (+ 2443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:31:03.088883 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:31:03.089003 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:31:03.093371 (+ 4368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:31:03.093380 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:31:03.093387 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:31:03.093438 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:31:03.097318 (+ 3880us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:31:03.098300 (+ 982us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:31:03.098309 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:31:03.098315 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:31:03.098483 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:31:03.098956 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:31:03.098963 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:31:03.098969 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:31:03.099378 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:31:03.099540 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 06:31:03.100371 (+ 831us) server_negotiation.cc:299] Negotiation successful
0811 06:31:03.100647 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":222,"threads_started":1}
I20260811 06:31:03.102005 13801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:31:03.079741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:42313 (local address 127.28.239.193:37325)
0811 06:31:03.080615 (+ 874us) negotiation.cc:107] Waiting for socket to connect
0811 06:31:03.080661 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 06:31:03.082750 (+ 2089us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:31:03.083689 (+ 939us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:31:03.083698 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:31:03.084248 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:31:03.084854 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:31:03.084874 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:31:03.086079 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:31:03.086083 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:31:03.088172 (+ 2089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:31:03.088187 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:31:03.089137 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:31:03.089147 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:31:03.089427 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:31:03.090316 (+ 889us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:31:03.090347 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:31:03.093125 (+ 2778us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:31:03.097516 (+ 4391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:31:03.097524 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:31:03.097615 (+ 91us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:31:03.098161 (+ 546us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:31:03.098617 (+ 456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:31:03.098620 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:31:03.098622 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:31:03.098823 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:31:03.099528 (+ 705us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:31:03.099534 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:31:03.099927 (+ 393us) client_negotiation.cc:769] Sending connection context
0811 06:31:03.100203 (+ 276us) client_negotiation.cc:240] Negotiation successful
0811 06:31:03.100648 (+ 445us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":607,"thread_start_us":370,"threads_started":1}
I20260811 06:31:03.103974 13797 heartbeater.cc:347] Connected to a master server at 127.28.239.254:42313
I20260811 06:31:03.104363 13797 heartbeater.cc:464] Registering TS with master...
I20260811 06:31:03.105165 13797 heartbeater.cc:511] Master 127.28.239.254:42313 requested a full tablet report, sending...
I20260811 06:31:03.107599 13482 ts_manager.cc:194] Registered new tserver with Master: 330964ae27904a7388a16ff44e34f187 (127.28.239.193:41665)
I20260811 06:31:03.109133 13482 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.239.193@KRBTEST.COM'} at 127.28.239.193:37325
WARNING: no policy specified for kudu/127.28.239.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.239.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.239.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.194.keytab.
Entry for principal kudu/127.28.239.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.194.keytab.
I20260811 06:31:03.158288 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.194:0
--local_ip_for_outbound_sockets=127.28.239.194
--webserver_interface=127.28.239.194
--webserver_port=0
--tserver_master_addrs=127.28.239.254:42313
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.194.keytab
--principal=kudu/127.28.239.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:45063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:31:03.515583 13807 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:31:03.515944 13807 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:31:03.516129 13807 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:31:03.523788 13807 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:31:03.523959 13807 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:31:03.524207 13807 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.194
I20260811 06:31:03.534646 13807 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:45063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.194.keytab
--principal=kudu/127.28.239.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.239.194
--webserver_port=0
--tserver_master_addrs=127.28.239.254:42313
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.194
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:31:03.536111 13807 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:31:03.537757 13807 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:31:03.552053 13815 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:31:03.551940 13816 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:31:03.552022 13821 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:31:03.552873 13807 server_base.cc:1034] running on GCE node
I20260811 06:31:03.553609 13807 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:31:03.554621 13807 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:31:03.555895 13807 hybrid_clock.cc:648] HybridClock initialized: now 1786429863555845 us; error 42 us; skew 500 ppm
I20260811 06:31:03.556296 13807 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:31:03 dist-test-slave-gkw7 krb5kdc[13415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429863, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:31:03.559640 13807 init.cc:377] Logged in from keytab as kudu/127.28.239.194@KRBTEST.COM (short username kudu)
I20260811 06:31:03.561331 13807 webserver.cc:460] Webserver started at http://127.28.239.194:34787/ using document root <none> and password file <none>
I20260811 06:31:03.562162 13807 fs_manager.cc:359] Metadata directory not provided
I20260811 06:31:03.562322 13807 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:31:03.562641 13807 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:31:03.565758 13807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/minicluster-data/ts-1/data/instance:
uuid: "8661b9535c8a44fd8abb6823728ce6fc"
format_stamp: "Formatted at 2026-08-11 06:31:03 on dist-test-slave-gkw7"
server_key: "727e4b42f597c602082b03837d683681"
server_key_iv: "bcf5d1fd7b1f6732aa0e749d6f399873"
server_key_version: "encryptionkey@0"
I20260811 06:31:03.566593 13807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/minicluster-data/ts-1/wal/instance:
uuid: "8661b9535c8a44fd8abb6823728ce6fc"
format_stamp: "Formatted at 2026-08-11 06:31:03 on dist-test-slave-gkw7"
server_key: "727e4b42f597c602082b03837d683681"
server_key_iv: "bcf5d1fd7b1f6732aa0e749d6f399873"
server_key_version: "encryptionkey@0"
I20260811 06:31:03.572008 13807 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 06:31:03.575798 13833 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:31:03.577181 13807 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 06:31:03.577370 13807 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/minicluster-data/ts-1/wal
uuid: "8661b9535c8a44fd8abb6823728ce6fc"
format_stamp: "Formatted at 2026-08-11 06:31:03 on dist-test-slave-gkw7"
server_key: "727e4b42f597c602082b03837d683681"
server_key_iv: "bcf5d1fd7b1f6732aa0e749d6f399873"
server_key_version: "encryptionkey@0"
I20260811 06:31:03.577615 13807 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:31:03.593317 13807 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:31:03.597872 13807 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:31:03.598215 13807 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:31:03.599335 13807 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:31:03.600759 13807 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:31:03.600863 13807 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:31:03.601024 13807 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:31:03.601111 13807 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:31:03.637990 13807 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.194:40331
I20260811 06:31:03.638042 13996 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.194:40331 every 8 connection(s)
I20260811 06:31:03.639730 13807 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
I20260811 06:31:03.640308 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 13807
I20260811 06:31:03.640532 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/minicluster-data/ts-1/wal/instance
I20260811 06:31:03.641191 29631 external_mini_cluster.cc:1428] Setting key 58546168dfbdec28220129a957421cab
Aug 11 06:31:03 dist-test-slave-gkw7 krb5kdc[13415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429863, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.194@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:31:03.664960 14000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:31:03.644420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.194:57635 (local address 127.28.239.254:42313)
0811 06:31:03.644917 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0811 06:31:03.644921 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:31:03.646332 (+ 1411us) server_negotiation.cc:407] Connection header received
0811 06:31:03.648407 (+ 2075us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:31:03.648412 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:31:03.648500 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:31:03.648644 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:31:03.650336 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:31:03.651077 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:31:03.652677 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:31:03.653119 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:31:03.653261 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:31:03.657278 (+ 4017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:31:03.657285 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:31:03.657291 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:31:03.657339 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:31:03.661250 (+ 3911us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:31:03.662154 (+ 904us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:31:03.662162 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:31:03.662168 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:31:03.662344 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:31:03.662813 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:31:03.662820 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:31:03.662826 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:31:03.663221 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:31:03.663380 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 06:31:03.664103 (+ 723us) server_negotiation.cc:299] Negotiation successful
0811 06:31:03.664439 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":300,"threads_started":1}
I20260811 06:31:03.665637 14003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:31:03.645205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:42313 (local address 127.28.239.194:57635)
0811 06:31:03.646163 (+ 958us) negotiation.cc:107] Waiting for socket to connect
0811 06:31:03.646204 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 06:31:03.648125 (+ 1921us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:31:03.648913 (+ 788us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:31:03.648922 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:31:03.649466 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:31:03.650155 (+ 689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:31:03.650174 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:31:03.651256 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:31:03.651260 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:31:03.652524 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:31:03.652537 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:31:03.653362 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:31:03.653369 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:31:03.653692 (+ 323us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:31:03.654434 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:31:03.654458 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:31:03.657001 (+ 2543us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:31:03.661422 (+ 4421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:31:03.661429 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:31:03.661442 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:31:03.662006 (+ 564us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:31:03.662475 (+ 469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:31:03.662478 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:31:03.662481 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:31:03.662676 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:31:03.663349 (+ 673us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:31:03.663355 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:31:03.663692 (+ 337us) client_negotiation.cc:769] Sending connection context
0811 06:31:03.663927 (+ 235us) client_negotiation.cc:240] Negotiation successful
0811 06:31:03.664367 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":637,"thread_start_us":411,"threads_started":1}
I20260811 06:31:03.667305 13997 heartbeater.cc:347] Connected to a master server at 127.28.239.254:42313
I20260811 06:31:03.667659 13997 heartbeater.cc:464] Registering TS with master...
I20260811 06:31:03.668466 13997 heartbeater.cc:511] Master 127.28.239.254:42313 requested a full tablet report, sending...
I20260811 06:31:03.670125 13482 ts_manager.cc:194] Registered new tserver with Master: 8661b9535c8a44fd8abb6823728ce6fc (127.28.239.194:40331)
I20260811 06:31:03.670989 13482 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.239.194@KRBTEST.COM'} at 127.28.239.194:57635
WARNING: no policy specified for kudu/127.28.239.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.239.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.239.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.195.keytab.
Entry for principal kudu/127.28.239.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.195.keytab.
I20260811 06:31:03.713521 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.195:0
--local_ip_for_outbound_sockets=127.28.239.195
--webserver_interface=127.28.239.195
--webserver_port=0
--tserver_master_addrs=127.28.239.254:42313
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.195.keytab
--principal=kudu/127.28.239.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:45063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:31:04.061769 14006 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:31:04.062076 14006 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:31:04.062242 14006 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:31:04.069579 14006 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:31:04.069697 14006 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:31:04.069881 14006 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.195
I20260811 06:31:04.079918 14006 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:45063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.195.keytab
--principal=kudu/127.28.239.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.239.195
--webserver_port=0
--tserver_master_addrs=127.28.239.254:42313
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.195
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:31:04.081319 14006 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:31:04.082790 14006 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:31:04.096402 14013 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:31:04.096365 14015 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:31:04.096429 14017 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:31:04.097674 14006 server_base.cc:1034] running on GCE node
I20260811 06:31:04.098284 14006 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:31:04.099261 14006 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:31:04.100509 14006 hybrid_clock.cc:648] HybridClock initialized: now 1786429864100437 us; error 56 us; skew 500 ppm
I20260811 06:31:04.100903 14006 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:31:04 dist-test-slave-gkw7 krb5kdc[13415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429864, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:31:04.107950 14006 init.cc:377] Logged in from keytab as kudu/127.28.239.195@KRBTEST.COM (short username kudu)
I20260811 06:31:04.109923 14006 webserver.cc:460] Webserver started at http://127.28.239.195:43705/ using document root <none> and password file <none>
I20260811 06:31:04.110739 14006 fs_manager.cc:359] Metadata directory not provided
I20260811 06:31:04.110860 14006 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:31:04.111174 14006 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:31:04.112238 13797 heartbeater.cc:503] Master 127.28.239.254:42313 was elected leader, sending a full tablet report...
I20260811 06:31:04.115146 14006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/minicluster-data/ts-2/data/instance:
uuid: "77bff554aede4f38ab6116c22d484b51"
format_stamp: "Formatted at 2026-08-11 06:31:04 on dist-test-slave-gkw7"
server_key: "15e66459f2291cc539ca0c8aa7e7818d"
server_key_iv: "5cd484945a8f074fc7c08236a9099a8c"
server_key_version: "encryptionkey@0"
I20260811 06:31:04.116124 14006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/minicluster-data/ts-2/wal/instance:
uuid: "77bff554aede4f38ab6116c22d484b51"
format_stamp: "Formatted at 2026-08-11 06:31:04 on dist-test-slave-gkw7"
server_key: "15e66459f2291cc539ca0c8aa7e7818d"
server_key_iv: "5cd484945a8f074fc7c08236a9099a8c"
server_key_version: "encryptionkey@0"
I20260811 06:31:04.122570 14006 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260811 06:31:04.126806 14028 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:31:04.128393 14006 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.002s
I20260811 06:31:04.128607 14006 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/minicluster-data/ts-2/wal
uuid: "77bff554aede4f38ab6116c22d484b51"
format_stamp: "Formatted at 2026-08-11 06:31:04 on dist-test-slave-gkw7"
server_key: "15e66459f2291cc539ca0c8aa7e7818d"
server_key_iv: "5cd484945a8f074fc7c08236a9099a8c"
server_key_version: "encryptionkey@0"
I20260811 06:31:04.128839 14006 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:31:04.152689 14006 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:31:04.157610 14006 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:31:04.157944 14006 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:31:04.159046 14006 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:31:04.160506 14006 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:31:04.160619 14006 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:31:04.160773 14006 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:31:04.160851 14006 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:31:04.197830 14006 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.195:39597
I20260811 06:31:04.197927 14201 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.195:39597 every 8 connection(s)
I20260811 06:31:04.199241 14006 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
I20260811 06:31:04.204897 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 14006
I20260811 06:31:04.205111 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/minicluster-data/ts-2/wal/instance
I20260811 06:31:04.205622 29631 external_mini_cluster.cc:1428] Setting key 3fcc4e73d80336ef13e026a08dcdaba7
Aug 11 06:31:04 dist-test-slave-gkw7 krb5kdc[13415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429864, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.195@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:31:04.221706 14206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:31:04.202071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.195:56517 (local address 127.28.239.254:42313)
0811 06:31:04.202613 (+ 542us) server_negotiation.cc:206] Beginning negotiation
0811 06:31:04.202618 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:31:04.203715 (+ 1097us) server_negotiation.cc:407] Connection header received
0811 06:31:04.205859 (+ 2144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:31:04.205867 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:31:04.205979 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:31:04.206111 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:31:04.208212 (+ 2101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:31:04.209253 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:31:04.210959 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:31:04.211385 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:31:04.211525 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:31:04.215380 (+ 3855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:31:04.215385 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:31:04.215388 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:31:04.215422 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:31:04.218172 (+ 2750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:31:04.218964 (+ 792us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:31:04.218968 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:31:04.218970 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:31:04.219108 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:31:04.219606 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:31:04.219612 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:31:04.219616 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:31:04.219972 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:31:04.220151 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0811 06:31:04.220998 (+ 847us) server_negotiation.cc:299] Negotiation successful
0811 06:31:04.221245 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":419,"thread_start_us":272,"threads_started":1}
I20260811 06:31:04.222703 14208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:31:04.202694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:42313 (local address 127.28.239.195:56517)
0811 06:31:04.203517 (+ 823us) negotiation.cc:107] Waiting for socket to connect
0811 06:31:04.203568 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 06:31:04.205465 (+ 1897us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:31:04.206528 (+ 1063us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:31:04.206538 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:31:04.207262 (+ 724us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:31:04.208007 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:31:04.208029 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:31:04.209426 (+ 1397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:31:04.209431 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:31:04.210804 (+ 1373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:31:04.210817 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:31:04.211634 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:31:04.211642 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:31:04.211899 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:31:04.212646 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:31:04.212669 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:31:04.215144 (+ 2475us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:31:04.218330 (+ 3186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:31:04.218340 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:31:04.218357 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:31:04.218821 (+ 464us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:31:04.219232 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:31:04.219237 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:31:04.219240 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:31:04.219440 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:31:04.220117 (+ 677us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:31:04.220125 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:31:04.220542 (+ 417us) client_negotiation.cc:769] Sending connection context
0811 06:31:04.220901 (+ 359us) client_negotiation.cc:240] Negotiation successful
0811 06:31:04.221481 (+ 580us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":566,"thread_start_us":338,"threads_started":1}
I20260811 06:31:04.224277 14203 heartbeater.cc:347] Connected to a master server at 127.28.239.254:42313
I20260811 06:31:04.224612 14203 heartbeater.cc:464] Registering TS with master...
I20260811 06:31:04.225389 14203 heartbeater.cc:511] Master 127.28.239.254:42313 requested a full tablet report, sending...
I20260811 06:31:04.226939 13482 ts_manager.cc:194] Registered new tserver with Master: 77bff554aede4f38ab6116c22d484b51 (127.28.239.195:39597)
I20260811 06:31:04.227814 13482 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.239.195@KRBTEST.COM'} at 127.28.239.195:56517
I20260811 06:31:04.234310 29631 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 06:31:04.253449 14206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:31:04.239163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48498 (local address 127.28.239.254:42313)
0811 06:31:04.239362 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 06:31:04.239370 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:31:04.239870 (+ 500us) server_negotiation.cc:407] Connection header received
0811 06:31:04.240082 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:31:04.240088 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:31:04.240182 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:31:04.240290 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:31:04.241828 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:31:04.242597 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:31:04.243871 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:31:04.244300 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:31:04.244435 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:31:04.247122 (+ 2687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:31:04.247127 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:31:04.247130 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:31:04.247165 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:31:04.250387 (+ 3222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:31:04.251217 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:31:04.251231 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:31:04.251234 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:31:04.251367 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:31:04.251782 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:31:04.251788 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:31:04.251791 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:31:04.252158 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:31:04.252434 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0811 06:31:04.252837 (+ 403us) server_negotiation.cc:299] Negotiation successful
0811 06:31:04.253089 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 06:31:04.258065 13482 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48498:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:31:04.266461 13482 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:31:04.296106 14224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:31:04.281062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34854 (local address 127.28.239.194:40331)
0811 06:31:04.284032 (+ 2970us) server_negotiation.cc:206] Beginning negotiation
0811 06:31:04.284132 (+ 100us) server_negotiation.cc:399] Waiting for connection header
0811 06:31:04.284172 (+ 40us) server_negotiation.cc:407] Connection header received
0811 06:31:04.284302 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:31:04.284311 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:31:04.284660 (+ 349us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:31:04.284938 (+ 278us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:31:04.286400 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:31:04.288870 (+ 2470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:31:04.292807 (+ 3937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:31:04.294717 (+ 1910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:31:04.294926 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:31:04.295142 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0811 06:31:04.295568 (+ 426us) server_negotiation.cc:299] Negotiation successful
0811 06:31:04.295753 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2774,"thread_start_us":2653,"threads_started":1}
I20260811 06:31:04.297336 14219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:31:04.280288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.194:40331 (local address 127.0.0.1:34854)
0811 06:31:04.282622 (+ 2334us) negotiation.cc:107] Waiting for socket to connect
0811 06:31:04.282673 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 06:31:04.282907 (+ 234us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:31:04.285389 (+ 2482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:31:04.285402 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:31:04.285451 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:31:04.286033 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:31:04.286061 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:31:04.289142 (+ 3081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:31:04.289152 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:31:04.292563 (+ 3411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:31:04.292583 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:31:04.294931 (+ 2348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:31:04.294937 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:31:04.295160 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:31:04.295233 (+ 73us) client_negotiation.cc:769] Sending connection context
0811 06:31:04.296646 (+ 1413us) client_negotiation.cc:240] Negotiation successful
0811 06:31:04.296893 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2057,"thread_start_us":13622,"threads_started":1}
I20260811 06:31:04.302275 14225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:31:04.280407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.195:39597 (local address 127.0.0.1:39442)
0811 06:31:04.292037 (+ 11630us) negotiation.cc:107] Waiting for socket to connect
0811 06:31:04.292072 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 06:31:04.293079 (+ 1007us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:31:04.293963 (+ 884us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:31:04.293967 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:31:04.294004 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:31:04.294470 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:31:04.294482 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:31:04.296816 (+ 2334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:31:04.296822 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:31:04.299323 (+ 2501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:31:04.299337 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:31:04.301419 (+ 2082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:31:04.301425 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:31:04.301623 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:31:04.301659 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 06:31:04.301770 (+ 111us) client_negotiation.cc:240] Negotiation successful
0811 06:31:04.301926 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11428,"thread_start_us":1772,"threads_started":1}
I20260811 06:31:04.302500 14222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:31:04.280422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39442 (local address 127.28.239.195:39597)
0811 06:31:04.283574 (+ 3152us) server_negotiation.cc:206] Beginning negotiation
0811 06:31:04.283586 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 06:31:04.292218 (+ 8632us) server_negotiation.cc:407] Connection header received
0811 06:31:04.293267 (+ 1049us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:31:04.293279 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:31:04.293672 (+ 393us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:31:04.294679 (+ 1007us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:31:04.295000 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:31:04.296642 (+ 1642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:31:04.299515 (+ 2873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:31:04.301201 (+ 1686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:31:04.301432 (+ 231us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:31:04.301691 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0811 06:31:04.301888 (+ 197us) server_negotiation.cc:299] Negotiation successful
0811 06:31:04.302081 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2938,"thread_start_us":2754,"threads_started":1}
I20260811 06:31:04.302983 14218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:31:04.283753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.193:41665 (local address 127.0.0.1:51734)
0811 06:31:04.294270 (+ 10517us) negotiation.cc:107] Waiting for socket to connect
0811 06:31:04.294290 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 06:31:04.294532 (+ 242us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:31:04.295244 (+ 712us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:31:04.295249 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:31:04.295277 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:31:04.295689 (+ 412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:31:04.295703 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:31:04.297716 (+ 2013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:31:04.297721 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:31:04.299576 (+ 1855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:31:04.299591 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:31:04.301918 (+ 2327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:31:04.301925 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:31:04.302133 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:31:04.302170 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 06:31:04.302284 (+ 114us) client_negotiation.cc:240] Negotiation successful
0811 06:31:04.302425 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10382,"thread_start_us":7854,"threads_started":1}
I20260811 06:31:04.303154 14226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:31:04.284550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51734 (local address 127.28.239.193:41665)
0811 06:31:04.293743 (+ 9193us) server_negotiation.cc:206] Beginning negotiation
0811 06:31:04.293752 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:31:04.294406 (+ 654us) server_negotiation.cc:407] Connection header received
0811 06:31:04.294756 (+ 350us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:31:04.294765 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:31:04.295019 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:31:04.295251 (+ 232us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:31:04.295936 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:31:04.297530 (+ 1594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:31:04.299754 (+ 2224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:31:04.301346 (+ 1592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:31:04.301588 (+ 242us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:31:04.301757 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0811 06:31:04.302648 (+ 891us) server_negotiation.cc:299] Negotiation successful
0811 06:31:04.302827 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8957,"thread_start_us":9554,"threads_started":1}
I20260811 06:31:04.302556 13899 tablet_service.cc:1467] Processing CreateTablet for tablet d1a33797599b48429bc8af1801a06037 (DEFAULT_TABLE table=test-table [id=2857a97da542460bbb68188c3f9ef3c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:31:04.304939 13899 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1a33797599b48429bc8af1801a06037. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:31:04.306608 14094 tablet_service.cc:1467] Processing CreateTablet for tablet d1a33797599b48429bc8af1801a06037 (DEFAULT_TABLE table=test-table [id=2857a97da542460bbb68188c3f9ef3c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:31:04.307005 13694 tablet_service.cc:1467] Processing CreateTablet for tablet d1a33797599b48429bc8af1801a06037 (DEFAULT_TABLE table=test-table [id=2857a97da542460bbb68188c3f9ef3c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:31:04.308142 14094 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1a33797599b48429bc8af1801a06037. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:31:04.308539 13694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1a33797599b48429bc8af1801a06037. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:31:04.318581 14228 tablet_bootstrap.cc:492] T d1a33797599b48429bc8af1801a06037 P 8661b9535c8a44fd8abb6823728ce6fc: Bootstrap starting.
I20260811 06:31:04.322979 14228 tablet_bootstrap.cc:654] T d1a33797599b48429bc8af1801a06037 P 8661b9535c8a44fd8abb6823728ce6fc: Neither blocks nor log segments found. Creating new log.
I20260811 06:31:04.323264 14229 tablet_bootstrap.cc:492] T d1a33797599b48429bc8af1801a06037 P 77bff554aede4f38ab6116c22d484b51: Bootstrap starting.
I20260811 06:31:04.324844 14228 log.cc:824] T d1a33797599b48429bc8af1801a06037 P 8661b9535c8a44fd8abb6823728ce6fc: Log is configured to *not* fsync() on all Append() calls
I20260811 06:31:04.325839 14230 tablet_bootstrap.cc:492] T d1a33797599b48429bc8af1801a06037 P 330964ae27904a7388a16ff44e34f187: Bootstrap starting.
I20260811 06:31:04.327409 14229 tablet_bootstrap.cc:654] T d1a33797599b48429bc8af1801a06037 P 77bff554aede4f38ab6116c22d484b51: Neither blocks nor log segments found. Creating new log.
I20260811 06:31:04.328814 14229 log.cc:824] T d1a33797599b48429bc8af1801a06037 P 77bff554aede4f38ab6116c22d484b51: Log is configured to *not* fsync() on all Append() calls
I20260811 06:31:04.329699 14230 tablet_bootstrap.cc:654] T d1a33797599b48429bc8af1801a06037 P 330964ae27904a7388a16ff44e34f187: Neither blocks nor log segments found. Creating new log.
I20260811 06:31:04.329718 14228 tablet_bootstrap.cc:492] T d1a33797599b48429bc8af1801a06037 P 8661b9535c8a44fd8abb6823728ce6fc: No bootstrap required, opened a new log
I20260811 06:31:04.330054 14228 ts_tablet_manager.cc:1397] T d1a33797599b48429bc8af1801a06037 P 8661b9535c8a44fd8abb6823728ce6fc: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.006s
I20260811 06:31:04.331015 14230 log.cc:824] T d1a33797599b48429bc8af1801a06037 P 330964ae27904a7388a16ff44e34f187: Log is configured to *not* fsync() on all Append() calls
I20260811 06:31:04.332470 14229 tablet_bootstrap.cc:492] T d1a33797599b48429bc8af1801a06037 P 77bff554aede4f38ab6116c22d484b51: No bootstrap required, opened a new log
I20260811 06:31:04.332785 14229 ts_tablet_manager.cc:1397] T d1a33797599b48429bc8af1801a06037 P 77bff554aede4f38ab6116c22d484b51: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260811 06:31:04.334110 14230 tablet_bootstrap.cc:492] T d1a33797599b48429bc8af1801a06037 P 330964ae27904a7388a16ff44e34f187: No bootstrap required, opened a new log
I20260811 06:31:04.334385 14230 ts_tablet_manager.cc:1397] T d1a33797599b48429bc8af1801a06037 P 330964ae27904a7388a16ff44e34f187: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.006s
I20260811 06:31:04.342674 14230 raft_consensus.cc:348] T d1a33797599b48429bc8af1801a06037 P 330964ae27904a7388a16ff44e34f187 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77bff554aede4f38ab6116c22d484b51" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 39597 } } peers { permanent_uuid: "8661b9535c8a44fd8abb6823728ce6fc" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 40331 } } peers { permanent_uuid: "330964ae27904a7388a16ff44e34f187" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 41665 } }
I20260811 06:31:04.342465 14228 raft_consensus.cc:348] T d1a33797599b48429bc8af1801a06037 P 8661b9535c8a44fd8abb6823728ce6fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77bff554aede4f38ab6116c22d484b51" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 39597 } } peers { permanent_uuid: "8661b9535c8a44fd8abb6823728ce6fc" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 40331 } } peers { permanent_uuid: "330964ae27904a7388a16ff44e34f187" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 41665 } }
I20260811 06:31:04.343045 14230 raft_consensus.cc:374] T d1a33797599b48429bc8af1801a06037 P 330964ae27904a7388a16ff44e34f187 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:31:04.343045 14228 raft_consensus.cc:374] T d1a33797599b48429bc8af1801a06037 P 8661b9535c8a44fd8abb6823728ce6fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:31:04.343153 14228 raft_consensus.cc:729] T d1a33797599b48429bc8af1801a06037 P 8661b9535c8a44fd8abb6823728ce6fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8661b9535c8a44fd8abb6823728ce6fc, State: Initialized, Role: FOLLOWER
I20260811 06:31:04.343163 14230 raft_consensus.cc:729] T d1a33797599b48429bc8af1801a06037 P 330964ae27904a7388a16ff44e34f187 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 330964ae27904a7388a16ff44e34f187, State: Initialized, Role: FOLLOWER
I20260811 06:31:04.343820 14230 consensus_queue.cc:260] T d1a33797599b48429bc8af1801a06037 P 330964ae27904a7388a16ff44e34f187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77bff554aede4f38ab6116c22d484b51" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 39597 } } peers { permanent_uuid: "8661b9535c8a44fd8abb6823728ce6fc" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 40331 } } peers { permanent_uuid: "330964ae27904a7388a16ff44e34f187" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 41665 } }
I20260811 06:31:04.343950 14228 consensus_queue.cc:260] T d1a33797599b48429bc8af1801a06037 P 8661b9535c8a44fd8abb6823728ce6fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77bff554aede4f38ab6116c22d484b51" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 39597 } } peers { permanent_uuid: "8661b9535c8a44fd8abb6823728ce6fc" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 40331 } } peers { permanent_uuid: "330964ae27904a7388a16ff44e34f187" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 41665 } }
I20260811 06:31:04.345204 14229 raft_consensus.cc:348] T d1a33797599b48429bc8af1801a06037 P 77bff554aede4f38ab6116c22d484b51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77bff554aede4f38ab6116c22d484b51" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 39597 } } peers { permanent_uuid: "8661b9535c8a44fd8abb6823728ce6fc" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 40331 } } peers { permanent_uuid: "330964ae27904a7388a16ff44e34f187" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 41665 } }
I20260811 06:31:04.345916 14230 ts_tablet_manager.cc:1428] T d1a33797599b48429bc8af1801a06037 P 330964ae27904a7388a16ff44e34f187: Time spent starting tablet: real 0.011s user 0.011s sys 0.001s
I20260811 06:31:04.345903 14229 raft_consensus.cc:374] T d1a33797599b48429bc8af1801a06037 P 77bff554aede4f38ab6116c22d484b51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:31:04.346031 14229 raft_consensus.cc:729] T d1a33797599b48429bc8af1801a06037 P 77bff554aede4f38ab6116c22d484b51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77bff554aede4f38ab6116c22d484b51, State: Initialized, Role: FOLLOWER
I20260811 06:31:04.346909 14229 consensus_queue.cc:260] T d1a33797599b48429bc8af1801a06037 P 77bff554aede4f38ab6116c22d484b51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77bff554aede4f38ab6116c22d484b51" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 39597 } } peers { permanent_uuid: "8661b9535c8a44fd8abb6823728ce6fc" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 40331 } } peers { permanent_uuid: "330964ae27904a7388a16ff44e34f187" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 41665 } }
I20260811 06:31:04.348382 14203 heartbeater.cc:503] Master 127.28.239.254:42313 was elected leader, sending a full tablet report...
I20260811 06:31:04.349381 14229 ts_tablet_manager.cc:1428] T d1a33797599b48429bc8af1801a06037 P 77bff554aede4f38ab6116c22d484b51: Time spent starting tablet: real 0.016s user 0.017s sys 0.000s
I20260811 06:31:04.350495 13997 heartbeater.cc:503] Master 127.28.239.254:42313 was elected leader, sending a full tablet report...
I20260811 06:31:04.351264 14228 ts_tablet_manager.cc:1428] T d1a33797599b48429bc8af1801a06037 P 8661b9535c8a44fd8abb6823728ce6fc: Time spent starting tablet: real 0.021s user 0.014s sys 0.005s
W20260811 06:31:04.394142 13999 tablet.cc:2369] T d1a33797599b48429bc8af1801a06037 P 8661b9535c8a44fd8abb6823728ce6fc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:31:04.425631 14236 raft_consensus.cc:482] T d1a33797599b48429bc8af1801a06037 P 330964ae27904a7388a16ff44e34f187 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:31:04.425985 14236 raft_consensus.cc:504] T d1a33797599b48429bc8af1801a06037 P 330964ae27904a7388a16ff44e34f187 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77bff554aede4f38ab6116c22d484b51" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 39597 } } peers { permanent_uuid: "8661b9535c8a44fd8abb6823728ce6fc" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 40331 } } peers { permanent_uuid: "330964ae27904a7388a16ff44e34f187" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 41665 } }
I20260811 06:31:04.427848 14236 leader_election.cc:288] T d1a33797599b48429bc8af1801a06037 P 330964ae27904a7388a16ff44e34f187 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 77bff554aede4f38ab6116c22d484b51 (127.28.239.195:39597), 8661b9535c8a44fd8abb6823728ce6fc (127.28.239.194:40331)
I20260811 06:31:04.438112 14222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:31:04.428573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.193:60477 (local address 127.28.239.195:39597)
0811 06:31:04.428806 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0811 06:31:04.428812 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:31:04.429799 (+ 987us) server_negotiation.cc:407] Connection header received
0811 06:31:04.429987 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:31:04.429992 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:31:04.430065 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:31:04.430176 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:31:04.431076 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:31:04.432280 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:31:04.435523 (+ 3243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:31:04.436837 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:31:04.437045 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:31:04.437127 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 06:31:04.437663 (+ 536us) server_negotiation.cc:299] Negotiation successful
0811 06:31:04.437816 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260811 06:31:04.438288 14224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:31:04.428563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.193:41429 (local address 127.28.239.194:40331)
0811 06:31:04.428795 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0811 06:31:04.428801 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:31:04.429804 (+ 1003us) server_negotiation.cc:407] Connection header received
0811 06:31:04.429959 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:31:04.429964 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:31:04.430055 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:31:04.430176 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:31:04.431075 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:31:04.432255 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:31:04.435524 (+ 3269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:31:04.436916 (+ 1392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:31:04.437061 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:31:04.437202 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 06:31:04.437742 (+ 540us) server_negotiation.cc:299] Negotiation successful
0811 06:31:04.437899 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 06:31:04.438349 14240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:31:04.428437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.194:40331 (local address 127.28.239.193:41429)
0811 06:31:04.429653 (+ 1216us) negotiation.cc:107] Waiting for socket to connect
0811 06:31:04.429679 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 06:31:04.429799 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:31:04.430210 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:31:04.430215 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:31:04.430267 (+ 52us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:31:04.430891 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:31:04.430906 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:31:04.432411 (+ 1505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:31:04.432416 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:31:04.435309 (+ 2893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:31:04.435328 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:31:04.437365 (+ 2037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:31:04.437373 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:31:04.437515 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:31:04.437582 (+ 67us) client_negotiation.cc:769] Sending connection context
0811 06:31:04.437687 (+ 105us) client_negotiation.cc:240] Negotiation successful
0811 06:31:04.437834 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1061,"mutex_wait_us":153,"spinlock_wait_cycles":21888,"thread_start_us":593,"threads_started":1}
I20260811 06:31:04.438390 14239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:31:04.428412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.195:39597 (local address 127.28.239.193:60477)
0811 06:31:04.429490 (+ 1078us) negotiation.cc:107] Waiting for socket to connect
0811 06:31:04.429511 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 06:31:04.429848 (+ 337us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:31:04.430210 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:31:04.430215 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:31:04.430267 (+ 52us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:31:04.430891 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:31:04.430905 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:31:04.432413 (+ 1508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:31:04.432417 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:31:04.435309 (+ 2892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:31:04.435328 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:31:04.437218 (+ 1890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:31:04.437238 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:31:04.437427 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:31:04.437458 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 06:31:04.437645 (+ 187us) client_negotiation.cc:240] Negotiation successful
0811 06:31:04.437829 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":913,"mutex_wait_us":311,"spinlock_wait_cycles":52096,"thread_start_us":594,"threads_started":1}
I20260811 06:31:04.439649 13925 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1a33797599b48429bc8af1801a06037" candidate_uuid: "330964ae27904a7388a16ff44e34f187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8661b9535c8a44fd8abb6823728ce6fc" is_pre_election: true
I20260811 06:31:04.439710 14127 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1a33797599b48429bc8af1801a06037" candidate_uuid: "330964ae27904a7388a16ff44e34f187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77bff554aede4f38ab6116c22d484b51" is_pre_election: true
I20260811 06:31:04.440263 13925 raft_consensus.cc:2454] T d1a33797599b48429bc8af1801a06037 P 8661b9535c8a44fd8abb6823728ce6fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 330964ae27904a7388a16ff44e34f187 in term 0.
I20260811 06:31:04.440317 14127 raft_consensus.cc:2454] T d1a33797599b48429bc8af1801a06037 P 77bff554aede4f38ab6116c22d484b51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 330964ae27904a7388a16ff44e34f187 in term 0.
I20260811 06:31:04.441045 13606 leader_election.cc:302] T d1a33797599b48429bc8af1801a06037 P 330964ae27904a7388a16ff44e34f187 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 330964ae27904a7388a16ff44e34f187, 77bff554aede4f38ab6116c22d484b51; no voters:
I20260811 06:31:04.441486 14236 raft_consensus.cc:2781] T d1a33797599b48429bc8af1801a06037 P 330964ae27904a7388a16ff44e34f187 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:31:04.441704 14236 raft_consensus.cc:482] T d1a33797599b48429bc8af1801a06037 P 330964ae27904a7388a16ff44e34f187 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:31:04.441848 14236 raft_consensus.cc:3037] T d1a33797599b48429bc8af1801a06037 P 330964ae27904a7388a16ff44e34f187 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:31:04.444310 14236 raft_consensus.cc:504] T d1a33797599b48429bc8af1801a06037 P 330964ae27904a7388a16ff44e34f187 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77bff554aede4f38ab6116c22d484b51" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 39597 } } peers { permanent_uuid: "8661b9535c8a44fd8abb6823728ce6fc" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 40331 } } peers { permanent_uuid: "330964ae27904a7388a16ff44e34f187" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 41665 } }
I20260811 06:31:04.445034 14236 leader_election.cc:288] T d1a33797599b48429bc8af1801a06037 P 330964ae27904a7388a16ff44e34f187 [CANDIDATE]: Term 1 election: Requested vote from peers 77bff554aede4f38ab6116c22d484b51 (127.28.239.195:39597), 8661b9535c8a44fd8abb6823728ce6fc (127.28.239.194:40331)
I20260811 06:31:04.445603 13925 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1a33797599b48429bc8af1801a06037" candidate_uuid: "330964ae27904a7388a16ff44e34f187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8661b9535c8a44fd8abb6823728ce6fc"
I20260811 06:31:04.445590 14127 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1a33797599b48429bc8af1801a06037" candidate_uuid: "330964ae27904a7388a16ff44e34f187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77bff554aede4f38ab6116c22d484b51"
I20260811 06:31:04.445827 13925 raft_consensus.cc:3037] T d1a33797599b48429bc8af1801a06037 P 8661b9535c8a44fd8abb6823728ce6fc [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:31:04.445827 14127 raft_consensus.cc:3037] T d1a33797599b48429bc8af1801a06037 P 77bff554aede4f38ab6116c22d484b51 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:31:04.449200 14127 raft_consensus.cc:2454] T d1a33797599b48429bc8af1801a06037 P 77bff554aede4f38ab6116c22d484b51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 330964ae27904a7388a16ff44e34f187 in term 1.
I20260811 06:31:04.449200 13925 raft_consensus.cc:2454] T d1a33797599b48429bc8af1801a06037 P 8661b9535c8a44fd8abb6823728ce6fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 330964ae27904a7388a16ff44e34f187 in term 1.
I20260811 06:31:04.449709 13608 leader_election.cc:302] T d1a33797599b48429bc8af1801a06037 P 330964ae27904a7388a16ff44e34f187 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 330964ae27904a7388a16ff44e34f187, 8661b9535c8a44fd8abb6823728ce6fc; no voters:
I20260811 06:31:04.450099 14236 raft_consensus.cc:2781] T d1a33797599b48429bc8af1801a06037 P 330964ae27904a7388a16ff44e34f187 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:31:04.450688 14236 raft_consensus.cc:686] T d1a33797599b48429bc8af1801a06037 P 330964ae27904a7388a16ff44e34f187 [term 1 LEADER]: Becoming Leader. State: Replica: 330964ae27904a7388a16ff44e34f187, State: Running, Role: LEADER
W20260811 06:31:04.450946 14205 tablet.cc:2369] T d1a33797599b48429bc8af1801a06037 P 77bff554aede4f38ab6116c22d484b51: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:31:04.451339 14236 consensus_queue.cc:237] T d1a33797599b48429bc8af1801a06037 P 330964ae27904a7388a16ff44e34f187 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77bff554aede4f38ab6116c22d484b51" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 39597 } } peers { permanent_uuid: "8661b9535c8a44fd8abb6823728ce6fc" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 40331 } } peers { permanent_uuid: "330964ae27904a7388a16ff44e34f187" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 41665 } }
I20260811 06:31:04.456228 13477 catalog_manager.cc:5495] T d1a33797599b48429bc8af1801a06037 P 330964ae27904a7388a16ff44e34f187 reported cstate change: term changed from 0 to 1, leader changed from <none> to 330964ae27904a7388a16ff44e34f187 (127.28.239.193). New cstate: current_term: 1 leader_uuid: "330964ae27904a7388a16ff44e34f187" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77bff554aede4f38ab6116c22d484b51" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 39597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8661b9535c8a44fd8abb6823728ce6fc" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 40331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "330964ae27904a7388a16ff44e34f187" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 41665 } health_report { overall_health: HEALTHY } } }
I20260811 06:31:04.496966 14226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:31:04.491562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51744 (local address 127.28.239.193:41665)
0811 06:31:04.491811 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0811 06:31:04.491818 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:31:04.491839 (+ 21us) server_negotiation.cc:407] Connection header received
0811 06:31:04.491992 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:31:04.491997 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:31:04.492099 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:31:04.492267 (+ 168us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:31:04.492658 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:31:04.493393 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:31:04.494777 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:31:04.495206 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:31:04.495345 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:31:04.495847 (+ 502us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:31:04.496377 (+ 530us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:31:04.496559 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0811 06:31:04.496612 (+ 53us) server_negotiation.cc:299] Negotiation successful
0811 06:31:04.496697 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 06:31:04.506258 13925 raft_consensus.cc:1261] T d1a33797599b48429bc8af1801a06037 P 8661b9535c8a44fd8abb6823728ce6fc [term 1 FOLLOWER]: Refusing update from remote peer 330964ae27904a7388a16ff44e34f187: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:31:04.506258 14127 raft_consensus.cc:1261] T d1a33797599b48429bc8af1801a06037 P 77bff554aede4f38ab6116c22d484b51 [term 1 FOLLOWER]: Refusing update from remote peer 330964ae27904a7388a16ff44e34f187: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:31:04.507406 14241 consensus_queue.cc:1035] T d1a33797599b48429bc8af1801a06037 P 330964ae27904a7388a16ff44e34f187 [LEADER]: Connected to new peer: Peer: permanent_uuid: "77bff554aede4f38ab6116c22d484b51" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 39597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:31:04.507776 14236 consensus_queue.cc:1035] T d1a33797599b48429bc8af1801a06037 P 330964ae27904a7388a16ff44e34f187 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8661b9535c8a44fd8abb6823728ce6fc" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 40331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:31:04.538444 13481 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48498:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:31:04.538802 13481 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48498:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:31:04.543428 13481 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f7db7631b20b4aaf96431f66aa172ecf: Sending DeleteTablet for 3 replicas of tablet d1a33797599b48429bc8af1801a06037
I20260811 06:31:04.544909 13899 tablet_service.cc:1514] Processing DeleteTablet for tablet d1a33797599b48429bc8af1801a06037 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:31:04 UTC) from {username='kudu', principal='kudu/127.28.239.254@KRBTEST.COM'} at 127.0.0.1:34854
I20260811 06:31:04.544937 13694 tablet_service.cc:1514] Processing DeleteTablet for tablet d1a33797599b48429bc8af1801a06037 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:31:04 UTC) from {username='kudu', principal='kudu/127.28.239.254@KRBTEST.COM'} at 127.0.0.1:51734
I20260811 06:31:04.544929 14094 tablet_service.cc:1514] Processing DeleteTablet for tablet d1a33797599b48429bc8af1801a06037 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:31:04 UTC) from {username='kudu', principal='kudu/127.28.239.254@KRBTEST.COM'} at 127.0.0.1:39442
I20260811 06:31:04.549295 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 13578
I20260811 06:31:04.581418 14256 tablet_replica.cc:330] T d1a33797599b48429bc8af1801a06037 P 77bff554aede4f38ab6116c22d484b51: stopping tablet replica
I20260811 06:31:04.582020 14256 raft_consensus.cc:2229] T d1a33797599b48429bc8af1801a06037 P 77bff554aede4f38ab6116c22d484b51 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:31:04.582428 14255 tablet_replica.cc:330] T d1a33797599b48429bc8af1801a06037 P 8661b9535c8a44fd8abb6823728ce6fc: stopping tablet replica
I20260811 06:31:04.582626 14256 raft_consensus.cc:2258] T d1a33797599b48429bc8af1801a06037 P 77bff554aede4f38ab6116c22d484b51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:31:04.583158 14255 raft_consensus.cc:2229] T d1a33797599b48429bc8af1801a06037 P 8661b9535c8a44fd8abb6823728ce6fc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:31:04.583715 14255 raft_consensus.cc:2258] T d1a33797599b48429bc8af1801a06037 P 8661b9535c8a44fd8abb6823728ce6fc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:31:04.585062 14259 tablet_replica.cc:330] T d1a33797599b48429bc8af1801a06037 P 330964ae27904a7388a16ff44e34f187: stopping tablet replica
I20260811 06:31:04.585515 14259 raft_consensus.cc:2229] T d1a33797599b48429bc8af1801a06037 P 330964ae27904a7388a16ff44e34f187 [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:31:04.586750 14255 ts_tablet_manager.cc:1905] T d1a33797599b48429bc8af1801a06037 P 8661b9535c8a44fd8abb6823728ce6fc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:31:04.586397 14259 raft_consensus.cc:2258] T d1a33797599b48429bc8af1801a06037 P 330964ae27904a7388a16ff44e34f187 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:31:04.588996 14259 ts_tablet_manager.cc:1905] T d1a33797599b48429bc8af1801a06037 P 330964ae27904a7388a16ff44e34f187: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 11 06:31:04 dist-test-slave-gkw7 krb5kdc[13415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429861, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.239.193@KRBTEST.COM
I20260811 06:31:04.591589 14256 ts_tablet_manager.cc:1905] T d1a33797599b48429bc8af1801a06037 P 77bff554aede4f38ab6116c22d484b51: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:31:04.595118 14255 ts_tablet_manager.cc:1918] T d1a33797599b48429bc8af1801a06037 P 8661b9535c8a44fd8abb6823728ce6fc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:31:04.595309 14255 log.cc:1196] T d1a33797599b48429bc8af1801a06037 P 8661b9535c8a44fd8abb6823728ce6fc: Deleting WAL directory at /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/minicluster-data/ts-1/wal/wals/d1a33797599b48429bc8af1801a06037
I20260811 06:31:04.595927 14255 ts_tablet_manager.cc:1939] T d1a33797599b48429bc8af1801a06037 P 8661b9535c8a44fd8abb6823728ce6fc: Deleting consensus metadata
I20260811 06:31:04.597646 14259 ts_tablet_manager.cc:1918] T d1a33797599b48429bc8af1801a06037 P 330964ae27904a7388a16ff44e34f187: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:31:04.597891 14259 log.cc:1196] T d1a33797599b48429bc8af1801a06037 P 330964ae27904a7388a16ff44e34f187: Deleting WAL directory at /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/minicluster-data/ts-0/wal/wals/d1a33797599b48429bc8af1801a06037
I20260811 06:31:04.598507 13464 catalog_manager.cc:4841] TS 8661b9535c8a44fd8abb6823728ce6fc (127.28.239.194:40331): tablet d1a33797599b48429bc8af1801a06037 (table test-table [id=2857a97da542460bbb68188c3f9ef3c4]) successfully deleted
I20260811 06:31:04.598980 14259 ts_tablet_manager.cc:1939] T d1a33797599b48429bc8af1801a06037 P 330964ae27904a7388a16ff44e34f187: Deleting consensus metadata
I20260811 06:31:04.600868 13458 catalog_manager.cc:4841] TS 330964ae27904a7388a16ff44e34f187 (127.28.239.193:41665): tablet d1a33797599b48429bc8af1801a06037 (table test-table [id=2857a97da542460bbb68188c3f9ef3c4]) successfully deleted
I20260811 06:31:04.604126 14256 ts_tablet_manager.cc:1918] T d1a33797599b48429bc8af1801a06037 P 77bff554aede4f38ab6116c22d484b51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:31:04.604348 14256 log.cc:1196] T d1a33797599b48429bc8af1801a06037 P 77bff554aede4f38ab6116c22d484b51: Deleting WAL directory at /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786429755927508-29631-0/minicluster-data/ts-2/wal/wals/d1a33797599b48429bc8af1801a06037
I20260811 06:31:04.604878 14256 ts_tablet_manager.cc:1939] T d1a33797599b48429bc8af1801a06037 P 77bff554aede4f38ab6116c22d484b51: Deleting consensus metadata
I20260811 06:31:04.606761 14226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:31:04.551041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51752 (local address 127.28.239.193:41665)
0811 06:31:04.551268 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0811 06:31:04.551275 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:31:04.571486 (+ 20211us) server_negotiation.cc:407] Connection header received
0811 06:31:04.572083 (+ 597us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:31:04.572091 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:31:04.572243 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:31:04.572341 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:31:04.574481 (+ 2140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:31:04.575943 (+ 1462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:31:04.584370 (+ 8427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:31:04.585203 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:31:04.585362 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:31:04.596850 (+ 11488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:31:04.596867 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:31:04.596888 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:31:04.596946 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:31:04.602014 (+ 5068us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:31:04.603265 (+ 1251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:31:04.603277 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:31:04.603282 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:31:04.603591 (+ 309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:31:04.604233 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:31:04.604239 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:31:04.604243 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:31:04.604760 (+ 517us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:31:04.605128 (+ 368us) server_negotiation.cc:1035] Waiting for connection context
0811 06:31:04.605792 (+ 664us) server_negotiation.cc:299] Negotiation successful
0811 06:31:04.606085 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260811 06:31:04.607136 13462 catalog_manager.cc:4841] TS 77bff554aede4f38ab6116c22d484b51 (127.28.239.195:39597): tablet d1a33797599b48429bc8af1801a06037 (table test-table [id=2857a97da542460bbb68188c3f9ef3c4]) successfully deleted
I20260811 06:31:04.607523 13784 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:31:04.718683 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 13578
I20260811 06:31:04.749984 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 13807
Aug 11 06:31:04 dist-test-slave-gkw7 krb5kdc[13415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429861, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.239.194@KRBTEST.COM
I20260811 06:31:04.766054 14224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:31:04.750841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34870 (local address 127.28.239.194:40331)
0811 06:31:04.751037 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 06:31:04.751042 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:31:04.751061 (+ 19us) server_negotiation.cc:407] Connection header received
0811 06:31:04.751278 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:31:04.751283 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:31:04.751372 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:31:04.751477 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:31:04.752736 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:31:04.753673 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:31:04.755103 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:31:04.755492 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:31:04.755598 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:31:04.759882 (+ 4284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:31:04.759898 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:31:04.759914 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:31:04.759950 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:31:04.763160 (+ 3210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:31:04.763814 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:31:04.763823 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:31:04.763825 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:31:04.763956 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:31:04.764384 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:31:04.764388 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:31:04.764390 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:31:04.764816 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:31:04.765046 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0811 06:31:04.765472 (+ 426us) server_negotiation.cc:299] Negotiation successful
0811 06:31:04.765739 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 06:31:04.766762 13990 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:31:04.858879 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 13807
I20260811 06:31:04.885664 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 14006
Aug 11 06:31:04 dist-test-slave-gkw7 krb5kdc[13415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429861, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.239.195@KRBTEST.COM
I20260811 06:31:04.901934 14222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:31:04.886629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39452 (local address 127.28.239.195:39597)
0811 06:31:04.886861 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0811 06:31:04.886869 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:31:04.886915 (+ 46us) server_negotiation.cc:407] Connection header received
0811 06:31:04.887179 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:31:04.887185 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:31:04.887263 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:31:04.887378 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:31:04.888769 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:31:04.889522 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:31:04.890856 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:31:04.891298 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:31:04.891400 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:31:04.896112 (+ 4712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:31:04.896129 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:31:04.896147 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:31:04.896189 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:31:04.899014 (+ 2825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:31:04.899744 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:31:04.899753 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:31:04.899755 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:31:04.899910 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:31:04.900300 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:31:04.900305 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:31:04.900309 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:31:04.900684 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:31:04.900998 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0811 06:31:04.901301 (+ 303us) server_negotiation.cc:299] Negotiation successful
0811 06:31:04.901555 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260811 06:31:04.902678 14195 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:31:04.993974 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 14006
I20260811 06:31:05.021461 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 13434
I20260811 06:31:05.022780 13534 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:31:05.136119 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 13434
2026-08-11T06:31:05Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4901 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:31:05 dist-test-slave-gkw7 krb5kdc[14272](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:31:05 dist-test-slave-gkw7 krb5kdc[14272](info): set up 2 sockets
Aug 11 06:31:05 dist-test-slave-gkw7 krb5kdc[14272](info): commencing operation
krb5kdc: starting...
W20260811 06:31:06.486343 29631 mini_kdc.cc:121] Time spent starting KDC: real 1.298s user 0.000s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/krb5kdc/test-user.keytab.
Aug 11 06:31:06 dist-test-slave-gkw7 krb5kdc[14272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429866, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:31:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:31:06Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.28.239.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.28.239.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.28.239.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/krb5kdc/rangeradmin_127.28.239.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.28.239.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/krb5kdc/rangeradmin_127.28.239.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.28.239.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.28.239.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.28.239.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/krb5kdc/rangerlookup_127.28.239.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.28.239.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/krb5kdc/rangerlookup_127.28.239.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.28.239.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.28.239.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.28.239.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/krb5kdc/HTTP_127.28.239.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.28.239.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/krb5kdc/HTTP_127.28.239.212@KRBTEST.COM.keytab.
I20260811 06:31:06.917279 29631 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskJveEHt/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/minicluster-data/postgres -l logfile start
2026-08-11 06:31:09.719 UTC [14324] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 06:31:09.720 UTC [14324] LOG: listening on IPv4 address "127.28.239.212", port 34163
2026-08-11 06:31:09.724 UTC [14324] LOG: listening on Unix socket "/tmp/.s.PGSQL.34163"
2026-08-11 06:31:09.751 UTC [14326] LOG: database system was shut down at 2026-08-11 06:31:08 UTC
2026-08-11 06:31:09.755 UTC [14324] LOG: database system is ready to accept connections
I20260811 06:31:11.046790 29631 mini_postgres.cc:96] Postgres bound to 34163
2026-08-11 06:31:11.075 UTC [14341] FATAL: database "slave" does not exist
127.28.239.212:34163 - accepting connections
I20260811 06:31:11.077718 29631 mini_ranger.cc:161] Starting Ranger...
I20260811 06:31:11.116665 29631 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 06:31:11.656136 29631 mini_ranger.cc:87] Created ranger Postgres database
I20260811 06:31:11.656311 29631 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/minicluster-data/ranger-admin
2026-08-11 06:31:12,039 [I] DB FLAVOR :POSTGRES
2026-08-11 06:31:12,040 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 06:31:12,040 [I] Checking connection..
2026-08-11 06:31:12,040 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJveEHt/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJveEHt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.239.212:34163/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 06:31:12,341 [I] Checking connection passed.
2026-08-11 06:31:12,341 [I] --------- Verifying version history table ---------
2026-08-11 06:31:12,341 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJveEHt/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJveEHt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.239.212:34163/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 06:31:12,601 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 06:31:12,602 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJveEHt/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJveEHt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.239.212:34163/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 06:31:12,864 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 06:31:12,864 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 06:31:12,864 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJveEHt/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJveEHt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.239.212:34163/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskJveEHt/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 06:31:13.111 UTC [14435] WARNING: there is no transaction in progress
2026-08-11 06:31:13,119 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 06:31:13,119 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJveEHt/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJveEHt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.239.212:34163/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 06:31:13,394 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 06:31:13,394 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 06:31:13,394 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJveEHt/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJveEHt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.239.212:34163/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 06:31:13,673 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJveEHt/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJveEHt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.239.212:34163/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 06:31:13,935 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJveEHt/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJveEHt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.239.212:34163/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-gkw7.c.gcp-upstream.internal','N') ;"
2026-08-11 06:31:14,218 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJveEHt/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJveEHt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.239.212:34163/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 06:31:14,503 [I] Table x_portal_user does not exist in database ranger
2026-08-11 06:31:14,504 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJveEHt/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJveEHt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.239.212:34163/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 06:31:14,757 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 06:31:14,757 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJveEHt/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJveEHt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.239.212:34163/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 06:31:15,033 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJveEHt/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJveEHt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.239.212:34163/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 06:31:15,328 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 06:31:15,328 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJveEHt/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJveEHt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.239.212:34163/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskJveEHt/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 06:31:16.412 UTC [14617] WARNING: there is no transaction in progress
2026-08-11 06:31:16.431 UTC [14617] WARNING: there is no transaction in progress
2026-08-11 06:31:16.451 UTC [14617] WARNING: there is no transaction in progress
2026-08-11 06:31:16.469 UTC [14617] WARNING: there is no transaction in progress
2026-08-11 06:31:16.486 UTC [14617] WARNING: there is no transaction in progress
2026-08-11 06:31:16.507 UTC [14617] WARNING: there is no transaction in progress
2026-08-11 06:31:16.566 UTC [14617] WARNING: there is no transaction in progress
2026-08-11 06:31:16.574 UTC [14617] WARNING: there is no transaction in progress
2026-08-11 06:31:16.590 UTC [14617] WARNING: there is no transaction in progress
2026-08-11 06:31:16.601 UTC [14617] WARNING: there is no transaction in progress
2026-08-11 06:31:16.612 UTC [14617] WARNING: there is no transaction in progress
2026-08-11 06:31:16.627 UTC [14617] WARNING: there is no transaction in progress
2026-08-11 06:31:16.637 UTC [14617] WARNING: there is no transaction in progress
2026-08-11 06:31:16.645 UTC [14617] WARNING: there is no transaction in progress
2026-08-11 06:31:16.652 UTC [14617] WARNING: there is no transaction in progress
2026-08-11 06:31:16.660 UTC [14617] WARNING: there is no transaction in progress
2026-08-11 06:31:17,424 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 06:31:17,425 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJveEHt/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJveEHt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.239.212:34163/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 06:31:17,693 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 06:31:17,693 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJveEHt/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJveEHt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.239.212:34163/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 06:31:17,993 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 06:31:17,994 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJveEHt/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJveEHt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.239.212:34163/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 06:31:18,274 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 06:31:18,274 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJveEHt/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJveEHt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.239.212:34163/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 06:31:18,541 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJveEHt/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJveEHt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.239.212:34163/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 06:31:18,813 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJveEHt/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJveEHt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.239.212:34163/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-gkw7.c.gcp-upstream.internal';"
2026-08-11 06:31:19,086 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 06:31:19,086 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJveEHt/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJveEHt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.239.212:34163/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 06:31:19,371 [I] DB_PATCHES have already been applied
I20260811 06:31:19.377506 29631 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/minicluster-data/ranger-admin:/tmp/dist-test-taskJveEHt/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskJveEHt/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskJveEHt/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJveEHt/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 06:31:19.377717 29631 mini_ranger.cc:193] Using host: 127.28.239.212
I20260811 06:31:19.401867 29631 mini_ranger.cc:227] Ranger admin URL: http://127.28.239.212:46435
Aug 11, 2026 6:31:19 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 6:31:19 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 6:31:20 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 6:31:20 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 6:31:20 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 6:31:20 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 6:31:20 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 06:31:20.904701 29631 mini_ranger.cc:160] Time spent starting Ranger: real 9.827s user 0.003s sys 0.075s
Aug 11, 2026 6:31:42 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 6:31:42 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 11, 2026 6:31:42 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 6:31:42 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 6:31:42 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 6:31:42 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 6:31:42 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 6:31:42 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 6:31:42 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 6:31:42 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 6:31:42 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 6:31:42 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 6:31:42 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 6:31:42 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 6:31:42 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 6:31:42 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 6:31:42 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 6:31:43 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 06:31:44.497654 29631 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.28.239.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.28.239.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.28.239.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/krb5kdc/rangerkms_127.28.239.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.28.239.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/krb5kdc/rangerkms_127.28.239.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.28.239.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.28.239.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.28.239.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/krb5kdc/HTTP_127.28.239.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.28.239.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/krb5kdc/HTTP_127.28.239.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 06:31:44 dist-test-slave-gkw7 krb5kdc[14272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429904, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 06:31:44.756937 29631 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 06:31:44.788038 29631 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 06:31:45.749457 29631 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 06:31:45.749688 29631 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/minicluster-data/ranger-kms
I20260811 06:31:45.749810 29631 mini_ranger_kms.cc:227] Using postgres at 127.28.239.212:34163
2026-08-11 06:31:46,459 [I] DB FLAVOR :POSTGRES
2026-08-11 06:31:46,459 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 06:31:46,459 [I] Checking connection
2026-08-11 06:31:46,459 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJveEHt/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.239.212:34163/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 06:31:46,716 [I] connection success
2026-08-11 06:31:46,716 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 06:31:46,716 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 06:31:46,716 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJveEHt/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.239.212:34163/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 06:31:46,957 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 06:31:46,957 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 06:31:46,957 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 06:31:46,957 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJveEHt/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.239.212:34163/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 06:31:47,219 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 06:31:47.754225 29631 mini_ranger_kms.cc:310] Created kms service
I20260811 06:31:47.928921 29631 mini_ranger_kms.cc:326] Created kudu user
I20260811 06:31:47.975857 29631 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 06:31:47 dist-test-slave-gkw7 krb5kdc[14272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429904, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.28.239.212@KRBTEST.COM
I20260811 06:31:48.435657 29631 mini_ranger_kms.cc:384] Added ranger policy
I20260811 06:31:48.437084 29631 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/minicluster-data/ranger-kms:/tmp/dist-test-taskJveEHt/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJveEHt/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskJveEHt/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 06:31:48.437162 29631 mini_ranger_kms.cc:240] Using host: 127.28.239.212
I20260811 06:31:48.465381 29631 mini_ranger_kms.cc:276] Ranger KMS PID: 14902
I20260811 06:31:48.465694 29631 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.28.239.212:35765
Aug 11, 2026 6:31:49 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 6:31:49 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 6:31:49 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 6:31:49 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 6:31:50 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 6:31:50 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 11 06:31:50 dist-test-slave-gkw7 krb5kdc[14272](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786429910, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.28.239.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:31:51.355535 29631 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.599s user 0.122s sys 0.463s
I20260811 06:31:51.355942 29631 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 06:31:51.356182 29631 mini_ranger_kms.cc:401] 127.28.239.212:35765/kms/v1/keys
Aug 11 06:31:51 dist-test-slave-gkw7 krb5kdc[14272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429910, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.28.239.212@KRBTEST.COM for HTTP/127.28.239.212@KRBTEST.COM
Aug 11 06:31:51 dist-test-slave-gkw7 krb5kdc[14272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429910, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.28.239.212@KRBTEST.COM for HTTP/127.28.239.212@KRBTEST.COM
Aug 11 06:31:54 dist-test-slave-gkw7 krb5kdc[14272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429914, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.28.239.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.239.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.239.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab.
Entry for principal kudu/127.28.239.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab.
I20260811 06:31:54.301824 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:33707
--webserver_interface=127.28.239.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab
--principal=kudu/127.28.239.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:39397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.239.254:33707
--ranger_config_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.28.239.212:35765/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:31:54.662492 14982 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:31:54.662788 14982 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:31:54.662950 14982 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:31:54.670195 14982 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:31:54.670310 14982 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:31:54.670441 14982 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:31:54.670573 14982 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:31:54.670660 14982 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:31:54.682294 14982 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:39397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/minicluster-data/master-0/wal
--ranger_kms_url=127.28.239.212:35765/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.28.239.254:33707
--ranger_config_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab
--principal=kudu/127.28.239.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:33707
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.239.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:31:54.683848 14982 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:31:54.685282 14982 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:31:54.698650 14995 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:31:54.698697 14992 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:31:54.698601 14990 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:31:54.699900 14982 server_base.cc:1034] running on GCE node
I20260811 06:31:54.700735 14982 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:31:54.702286 14982 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:31:54.703577 14982 hybrid_clock.cc:648] HybridClock initialized: now 1786429914703535 us; error 29 us; skew 500 ppm
I20260811 06:31:54.703980 14982 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:31:54 dist-test-slave-gkw7 krb5kdc[14272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429914, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:31:54.707480 14982 init.cc:377] Logged in from keytab as kudu/127.28.239.254@KRBTEST.COM (short username kudu)
I20260811 06:31:54.709105 14982 webserver.cc:460] Webserver started at http://127.28.239.254:43411/ using document root <none> and password file <none>
I20260811 06:31:54.709882 14982 fs_manager.cc:359] Metadata directory not provided
I20260811 06:31:54.710014 14982 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:31:54.710352 14982 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 06:31:54 dist-test-slave-gkw7 krb5kdc[14272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429914, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.254@KRBTEST.COM for HTTP/127.28.239.212@KRBTEST.COM
I20260811 06:31:54.772859 14982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/minicluster-data/master-0/data/instance:
uuid: "e5486c2f2a504308aa5f91c8758b2411"
format_stamp: "Formatted at 2026-08-11 06:31:54 on dist-test-slave-gkw7"
server_key: "ea6c5b337043e9fb7c4e513dc4d9de9e"
server_key_iv: "eeccae8f3ed8b51abe22da93fed7c34c"
server_key_version: "kuduclusterkey@0"
I20260811 06:31:54.773845 14982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/minicluster-data/master-0/wal/instance:
uuid: "e5486c2f2a504308aa5f91c8758b2411"
format_stamp: "Formatted at 2026-08-11 06:31:54 on dist-test-slave-gkw7"
server_key: "ea6c5b337043e9fb7c4e513dc4d9de9e"
server_key_iv: "eeccae8f3ed8b51abe22da93fed7c34c"
server_key_version: "kuduclusterkey@0"
I20260811 06:31:54.779572 14982 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 06:31:54.802031 15003 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:31:54.803921 14982 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.003s
I20260811 06:31:54.804136 14982 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/minicluster-data/master-0/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/minicluster-data/master-0/wal
uuid: "e5486c2f2a504308aa5f91c8758b2411"
format_stamp: "Formatted at 2026-08-11 06:31:54 on dist-test-slave-gkw7"
server_key: "ea6c5b337043e9fb7c4e513dc4d9de9e"
server_key_iv: "eeccae8f3ed8b51abe22da93fed7c34c"
server_key_version: "kuduclusterkey@0"
I20260811 06:31:54.804406 14982 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:31:54.833271 14982 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:31:54.839054 14982 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:31:54.839416 14982 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:31:54.863087 15087 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.254:33707 every 8 connection(s)
I20260811 06:31:54.863123 14982 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.254:33707
I20260811 06:31:54.865176 14982 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
I20260811 06:31:54.869292 15088 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:31:54.869989 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 14982
I20260811 06:31:54.870217 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/minicluster-data/master-0/wal/instance
I20260811 06:31:54.871075 29631 external_mini_cluster.cc:1428] Setting key c04671195a69c3d156647b17eef3f4b4
I20260811 06:31:54.882720 15088 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5486c2f2a504308aa5f91c8758b2411: Bootstrap starting.
Aug 11 06:31:54 dist-test-slave-gkw7 krb5kdc[14272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429914, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:31:54.886437 15088 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e5486c2f2a504308aa5f91c8758b2411: Neither blocks nor log segments found. Creating new log.
I20260811 06:31:54.887883 15088 log.cc:824] T 00000000000000000000000000000000 P e5486c2f2a504308aa5f91c8758b2411: Log is configured to *not* fsync() on all Append() calls
I20260811 06:31:54.891201 15088 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5486c2f2a504308aa5f91c8758b2411: No bootstrap required, opened a new log
I20260811 06:31:54.894737 15091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:31:54.873425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60384 (local address 127.28.239.254:33707)
0811 06:31:54.874304 (+ 879us) server_negotiation.cc:206] Beginning negotiation
0811 06:31:54.874317 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 06:31:54.874373 (+ 56us) server_negotiation.cc:407] Connection header received
0811 06:31:54.876314 (+ 1941us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:31:54.876340 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:31:54.876792 (+ 452us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:31:54.877219 (+ 427us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:31:54.878697 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:31:54.880147 (+ 1450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:31:54.881560 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:31:54.882319 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:31:54.882552 (+ 233us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:31:54.886937 (+ 4385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:31:54.886946 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:31:54.886962 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:31:54.887009 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:31:54.890436 (+ 3427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:31:54.891157 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:31:54.891164 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:31:54.891171 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:31:54.891421 (+ 250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:31:54.891823 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:31:54.891828 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:31:54.891832 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:31:54.892370 (+ 538us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:31:54.892698 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0811 06:31:54.893218 (+ 520us) server_negotiation.cc:299] Negotiation successful
0811 06:31:54.893691 (+ 473us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":654,"thread_start_us":336,"threads_started":1}
I20260811 06:31:54.899525 15088 raft_consensus.cc:348] T 00000000000000000000000000000000 P e5486c2f2a504308aa5f91c8758b2411 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5486c2f2a504308aa5f91c8758b2411" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 33707 } }
I20260811 06:31:54.899914 15088 raft_consensus.cc:374] T 00000000000000000000000000000000 P e5486c2f2a504308aa5f91c8758b2411 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:31:54.900018 15088 raft_consensus.cc:729] T 00000000000000000000000000000000 P e5486c2f2a504308aa5f91c8758b2411 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5486c2f2a504308aa5f91c8758b2411, State: Initialized, Role: FOLLOWER
I20260811 06:31:54.900669 15088 consensus_queue.cc:260] T 00000000000000000000000000000000 P e5486c2f2a504308aa5f91c8758b2411 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5486c2f2a504308aa5f91c8758b2411" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 33707 } }
I20260811 06:31:54.900915 15088 raft_consensus.cc:388] T 00000000000000000000000000000000 P e5486c2f2a504308aa5f91c8758b2411 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:31:54.901077 15088 raft_consensus.cc:482] T 00000000000000000000000000000000 P e5486c2f2a504308aa5f91c8758b2411 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:31:54.901247 15088 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e5486c2f2a504308aa5f91c8758b2411 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:31:54.903743 15088 raft_consensus.cc:504] T 00000000000000000000000000000000 P e5486c2f2a504308aa5f91c8758b2411 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5486c2f2a504308aa5f91c8758b2411" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 33707 } }
I20260811 06:31:54.904268 15088 leader_election.cc:302] T 00000000000000000000000000000000 P e5486c2f2a504308aa5f91c8758b2411 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e5486c2f2a504308aa5f91c8758b2411; no voters:
I20260811 06:31:54.904863 15088 leader_election.cc:288] T 00000000000000000000000000000000 P e5486c2f2a504308aa5f91c8758b2411 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:31:54.905030 15094 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e5486c2f2a504308aa5f91c8758b2411 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:31:54.905786 15094 raft_consensus.cc:686] T 00000000000000000000000000000000 P e5486c2f2a504308aa5f91c8758b2411 [term 1 LEADER]: Becoming Leader. State: Replica: e5486c2f2a504308aa5f91c8758b2411, State: Running, Role: LEADER
I20260811 06:31:54.906442 15094 consensus_queue.cc:237] T 00000000000000000000000000000000 P e5486c2f2a504308aa5f91c8758b2411 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5486c2f2a504308aa5f91c8758b2411" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 33707 } }
I20260811 06:31:54.907066 15088 sys_catalog.cc:564] T 00000000000000000000000000000000 P e5486c2f2a504308aa5f91c8758b2411 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:31:54.915035 15088 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 06:31:54.915182 15096 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5486c2f2a504308aa5f91c8758b2411 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e5486c2f2a504308aa5f91c8758b2411. Latest consensus state: current_term: 1 leader_uuid: "e5486c2f2a504308aa5f91c8758b2411" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5486c2f2a504308aa5f91c8758b2411" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 33707 } } }
I20260811 06:31:54.915189 15095 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5486c2f2a504308aa5f91c8758b2411 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e5486c2f2a504308aa5f91c8758b2411" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5486c2f2a504308aa5f91c8758b2411" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 33707 } } }
I20260811 06:31:54.915720 15096 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5486c2f2a504308aa5f91c8758b2411 [sys.catalog]: This master's current role is: LEADER
I20260811 06:31:54.915812 15095 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5486c2f2a504308aa5f91c8758b2411 [sys.catalog]: This master's current role is: LEADER
I20260811 06:31:54.923578 15112 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:31:54.931519 15112 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:31:54.943500 15112 catalog_manager.cc:1347] Generated new cluster ID: 8a900442ac51470a94df80924cabe877
I20260811 06:31:54.943658 15112 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:31:54.967192 15112 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:31:54.969406 15112 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:31:54.990767 15112 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e5486c2f2a504308aa5f91c8758b2411: Generated new TSK 0
I20260811 06:31:54.992107 15112 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.239.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.239.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.239.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.193.keytab.
Entry for principal kudu/127.28.239.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.193.keytab.
I20260811 06:31:55.733953 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.28.239.212:35765/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.193:0
--local_ip_for_outbound_sockets=127.28.239.193
--webserver_interface=127.28.239.193
--webserver_port=0
--tserver_master_addrs=127.28.239.254:33707
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.193.keytab
--principal=kudu/127.28.239.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:39397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 06:31:56 dist-test-slave-gkw7 krb5kdc[14272](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786429916, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260811 06:31:56.168716 15153 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:31:56.169158 15153 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:31:56.169317 15153 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:31:56.178399 15153 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:31:56.178580 15153 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:31:56.178785 15153 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.193
I20260811 06:31:56.189771 15153 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:39397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.28.239.212:35765/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.193.keytab
--principal=kudu/127.28.239.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.239.193
--webserver_port=0
--tserver_master_addrs=127.28.239.254:33707
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.193
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:31:56.191324 15153 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:31:56.192739 15153 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:31:56.212787 15175 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:31:56.212566 15179 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:31:56.212656 15177 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:31:56.215981 15153 server_base.cc:1034] running on GCE node
I20260811 06:31:56.216845 15153 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:31:56.218206 15153 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:31:56.219549 15153 hybrid_clock.cc:648] HybridClock initialized: now 1786429916219471 us; error 61 us; skew 500 ppm
I20260811 06:31:56.219954 15153 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:31:56 dist-test-slave-gkw7 krb5kdc[14272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429916, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:31:56.224493 15153 init.cc:377] Logged in from keytab as kudu/127.28.239.193@KRBTEST.COM (short username kudu)
I20260811 06:31:56.236083 15153 webserver.cc:460] Webserver started at http://127.28.239.193:38601/ using document root <none> and password file <none>
I20260811 06:31:56.237166 15153 fs_manager.cc:359] Metadata directory not provided
I20260811 06:31:56.237330 15153 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:31:56.237741 15153 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 06:31:56 dist-test-slave-gkw7 krb5kdc[14272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429916, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.193@KRBTEST.COM for HTTP/127.28.239.212@KRBTEST.COM
I20260811 06:31:56.267081 15153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/minicluster-data/ts-0/data/instance:
uuid: "f4ac3d7319bc43769f4e2cc48adee3ff"
format_stamp: "Formatted at 2026-08-11 06:31:56 on dist-test-slave-gkw7"
server_key: "5d4e122425ac6191dd644a0a0a92e570"
server_key_iv: "271045f89ad46edbfc72381564071272"
server_key_version: "kuduclusterkey@0"
I20260811 06:31:56.269248 15153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/minicluster-data/ts-0/wal/instance:
uuid: "f4ac3d7319bc43769f4e2cc48adee3ff"
format_stamp: "Formatted at 2026-08-11 06:31:56 on dist-test-slave-gkw7"
server_key: "5d4e122425ac6191dd644a0a0a92e570"
server_key_iv: "271045f89ad46edbfc72381564071272"
server_key_version: "kuduclusterkey@0"
I20260811 06:31:56.286867 15153 fs_manager.cc:691] Time spent creating directory manager: real 0.017s user 0.005s sys 0.004s
I20260811 06:31:56.330427 15188 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:31:56.333281 15153 fs_manager.cc:725] Time spent opening block manager: real 0.017s user 0.005s sys 0.009s
I20260811 06:31:56.333540 15153 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/minicluster-data/ts-0/wal
uuid: "f4ac3d7319bc43769f4e2cc48adee3ff"
format_stamp: "Formatted at 2026-08-11 06:31:56 on dist-test-slave-gkw7"
server_key: "5d4e122425ac6191dd644a0a0a92e570"
server_key_iv: "271045f89ad46edbfc72381564071272"
server_key_version: "kuduclusterkey@0"
I20260811 06:31:56.333860 15153 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:31:56.362327 15153 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:31:56.367980 15153 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:31:56.368379 15153 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:31:56.369800 15153 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:31:56.371743 15153 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:31:56.371824 15153 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:31:56.371969 15153 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:31:56.372035 15153 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:31:56.475458 15153 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.193:43153
I20260811 06:31:56.475502 15353 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.193:43153 every 8 connection(s)
I20260811 06:31:56.477298 15153 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
I20260811 06:31:56.478667 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 15153
I20260811 06:31:56.478888 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/minicluster-data/ts-0/wal/instance
I20260811 06:31:56.479743 29631 external_mini_cluster.cc:1428] Setting key 7764380e0f864bbbf74e602020b8cf5a
Aug 11 06:31:56 dist-test-slave-gkw7 krb5kdc[14272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429916, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.193@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:31:56.518867 15355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:31:56.483126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.193:48547 (local address 127.28.239.254:33707)
0811 06:31:56.498311 (+ 15185us) server_negotiation.cc:206] Beginning negotiation
0811 06:31:56.498317 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:31:56.498338 (+ 21us) server_negotiation.cc:407] Connection header received
0811 06:31:56.498402 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:31:56.498408 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:31:56.498505 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:31:56.498623 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:31:56.501099 (+ 2476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:31:56.502418 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:31:56.505054 (+ 2636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:31:56.505846 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:31:56.505950 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:31:56.510986 (+ 5036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:31:56.510993 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:31:56.510998 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:31:56.511051 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:31:56.514852 (+ 3801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:31:56.515896 (+ 1044us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:31:56.515902 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:31:56.515906 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:31:56.516066 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:31:56.516603 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:31:56.516608 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:31:56.516611 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:31:56.517046 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:31:56.517176 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 06:31:56.518242 (+ 1066us) server_negotiation.cc:299] Negotiation successful
0811 06:31:56.518484 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15059,"thread_start_us":14929,"threads_started":1}
I20260811 06:31:56.521370 15357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:31:56.483963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:33707 (local address 127.28.239.193:48547)
0811 06:31:56.494682 (+ 10719us) negotiation.cc:107] Waiting for socket to connect
0811 06:31:56.494743 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 06:31:56.497382 (+ 2639us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:31:56.499136 (+ 1754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:31:56.499154 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:31:56.499906 (+ 752us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:31:56.500866 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:31:56.500897 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:31:56.502649 (+ 1752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:31:56.502654 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:31:56.504866 (+ 2212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:31:56.504881 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:31:56.506388 (+ 1507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:31:56.506396 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:31:56.506729 (+ 333us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:31:56.507610 (+ 881us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:31:56.507642 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:31:56.510700 (+ 3058us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:31:56.515074 (+ 4374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:31:56.515085 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:31:56.515104 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:31:56.515747 (+ 643us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:31:56.516221 (+ 474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:31:56.516226 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:31:56.516230 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:31:56.516476 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:31:56.517291 (+ 815us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:31:56.517300 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:31:56.517781 (+ 481us) client_negotiation.cc:769] Sending connection context
0811 06:31:56.519381 (+ 1600us) client_negotiation.cc:240] Negotiation successful
0811 06:31:56.519966 (+ 585us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9304,"thread_start_us":9403,"threads_started":1}
WARNING: no policy specified for kudu/127.28.239.194@KRBTEST.COM; defaulting to no policy
I20260811 06:31:56.523548 15354 heartbeater.cc:347] Connected to a master server at 127.28.239.254:33707
I20260811 06:31:56.523953 15354 heartbeater.cc:464] Registering TS with master...
I20260811 06:31:56.524887 15354 heartbeater.cc:511] Master 127.28.239.254:33707 requested a full tablet report, sending...
I20260811 06:31:56.527776 15035 ts_manager.cc:194] Registered new tserver with Master: f4ac3d7319bc43769f4e2cc48adee3ff (127.28.239.193:43153)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.239.194@KRBTEST.COM" created.
I20260811 06:31:56.529937 15035 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.239.193@KRBTEST.COM'} at 127.28.239.193:48547
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.239.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.194.keytab.
Entry for principal kudu/127.28.239.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.194.keytab.
I20260811 06:31:56.566177 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.28.239.212:35765/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.194:0
--local_ip_for_outbound_sockets=127.28.239.194
--webserver_interface=127.28.239.194
--webserver_port=0
--tserver_master_addrs=127.28.239.254:33707
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.194.keytab
--principal=kudu/127.28.239.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:39397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 06:31:56 dist-test-slave-gkw7 krb5kdc[14272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429916, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.254@KRBTEST.COM for HTTP/127.28.239.212@KRBTEST.COM
Aug 11 06:31:56 dist-test-slave-gkw7 krb5kdc[14272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429916, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.254@KRBTEST.COM for HTTP/127.28.239.212@KRBTEST.COM
W20260811 06:31:57.067126 15369 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:31:57.067613 15369 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:31:57.067793 15369 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:31:57.079648 15369 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:31:57.079836 15369 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:31:57.080056 15369 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.194
I20260811 06:31:57.096218 15369 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:39397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.28.239.212:35765/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.194.keytab
--principal=kudu/127.28.239.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.239.194
--webserver_port=0
--tserver_master_addrs=127.28.239.254:33707
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.194
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:31:57.098218 15369 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:31:57.100066 15369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:31:57.137589 15379 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:31:57.139094 15380 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:31:57.155488 15383 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:31:57.156046 15369 server_base.cc:1034] running on GCE node
I20260811 06:31:57.157087 15369 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:31:57.164219 15369 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:31:57.165582 15369 hybrid_clock.cc:648] HybridClock initialized: now 1786429917165483 us; error 103 us; skew 500 ppm
I20260811 06:31:57.166034 15369 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:31:57 dist-test-slave-gkw7 krb5kdc[14272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429917, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:31:57.170679 15369 init.cc:377] Logged in from keytab as kudu/127.28.239.194@KRBTEST.COM (short username kudu)
I20260811 06:31:57.222064 15369 webserver.cc:460] Webserver started at http://127.28.239.194:42181/ using document root <none> and password file <none>
I20260811 06:31:57.223398 15369 fs_manager.cc:359] Metadata directory not provided
I20260811 06:31:57.223579 15369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:31:57.224045 15369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 06:31:57 dist-test-slave-gkw7 krb5kdc[14272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429917, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.194@KRBTEST.COM for HTTP/127.28.239.212@KRBTEST.COM
I20260811 06:31:57.270689 15369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/minicluster-data/ts-1/data/instance:
uuid: "f23311a2e25846e399721d66af185492"
format_stamp: "Formatted at 2026-08-11 06:31:57 on dist-test-slave-gkw7"
server_key: "735f3fd858c762f29c23e83247eff95f"
server_key_iv: "02532ad51c96c62d48b1470c13bab9af"
server_key_version: "kuduclusterkey@0"
I20260811 06:31:57.290727 15369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/minicluster-data/ts-1/wal/instance:
uuid: "f23311a2e25846e399721d66af185492"
format_stamp: "Formatted at 2026-08-11 06:31:57 on dist-test-slave-gkw7"
server_key: "735f3fd858c762f29c23e83247eff95f"
server_key_iv: "02532ad51c96c62d48b1470c13bab9af"
server_key_version: "kuduclusterkey@0"
I20260811 06:31:57.312175 15369 fs_manager.cc:691] Time spent creating directory manager: real 0.021s user 0.010s sys 0.001s
I20260811 06:31:57.356071 15408 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:31:57.359162 15369 fs_manager.cc:725] Time spent opening block manager: real 0.011s user 0.006s sys 0.004s
I20260811 06:31:57.359488 15369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/minicluster-data/ts-1/wal
uuid: "f23311a2e25846e399721d66af185492"
format_stamp: "Formatted at 2026-08-11 06:31:57 on dist-test-slave-gkw7"
server_key: "735f3fd858c762f29c23e83247eff95f"
server_key_iv: "02532ad51c96c62d48b1470c13bab9af"
server_key_version: "kuduclusterkey@0"
I20260811 06:31:57.359865 15369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:31:57.406067 15369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:31:57.413064 15369 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:31:57.413514 15369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:31:57.421089 15369 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:31:57.429771 15369 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:31:57.429947 15369 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:31:57.430119 15369 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:31:57.430220 15369 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:31:57.537151 15354 heartbeater.cc:503] Master 127.28.239.254:33707 was elected leader, sending a full tablet report...
I20260811 06:31:57.601338 15369 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.194:42931
I20260811 06:31:57.601365 15565 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.194:42931 every 8 connection(s)
I20260811 06:31:57.602833 15369 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
I20260811 06:31:57.606148 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 15369
I20260811 06:31:57.606321 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/minicluster-data/ts-1/wal/instance
I20260811 06:31:57.606840 29631 external_mini_cluster.cc:1428] Setting key 597515f272ed48d8b609c2186dc5d375
Aug 11 06:31:57 dist-test-slave-gkw7 krb5kdc[14272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429917, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.194@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:31:57.639716 15568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:31:57.606178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.194:50679 (local address 127.28.239.254:33707)
0811 06:31:57.621285 (+ 15107us) server_negotiation.cc:206] Beginning negotiation
0811 06:31:57.621294 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:31:57.622463 (+ 1169us) server_negotiation.cc:407] Connection header received
0811 06:31:57.624150 (+ 1687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:31:57.624156 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:31:57.624265 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:31:57.624418 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:31:57.626355 (+ 1937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:31:57.627169 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:31:57.628870 (+ 1701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:31:57.629303 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:31:57.629452 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:31:57.633083 (+ 3631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:31:57.633089 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:31:57.633092 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:31:57.633127 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:31:57.636018 (+ 2891us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:31:57.636805 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:31:57.636810 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:31:57.636812 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:31:57.636942 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:31:57.637405 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:31:57.637409 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:31:57.637411 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:31:57.637759 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:31:57.637925 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0811 06:31:57.639052 (+ 1127us) server_negotiation.cc:299] Negotiation successful
0811 06:31:57.639317 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14913,"thread_start_us":14755,"threads_started":1}
I20260811 06:31:57.641026 15570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:31:57.607030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:33707 (local address 127.28.239.194:50679)
0811 06:31:57.621738 (+ 14708us) negotiation.cc:107] Waiting for socket to connect
0811 06:31:57.621826 (+ 88us) client_negotiation.cc:174] Beginning negotiation
0811 06:31:57.623824 (+ 1998us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:31:57.624694 (+ 870us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:31:57.624703 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:31:57.625183 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:31:57.626131 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:31:57.626156 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:31:57.627354 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:31:57.627359 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:31:57.628703 (+ 1344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:31:57.628726 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:31:57.629584 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:31:57.629594 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:31:57.629855 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:31:57.630595 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:31:57.630620 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:31:57.632836 (+ 2216us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:31:57.636172 (+ 3336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:31:57.636181 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:31:57.636200 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:31:57.636656 (+ 456us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:31:57.637056 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:31:57.637062 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:31:57.637065 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:31:57.637275 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:31:57.637905 (+ 630us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:31:57.637914 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:31:57.638563 (+ 649us) client_negotiation.cc:769] Sending connection context
0811 06:31:57.638927 (+ 364us) client_negotiation.cc:240] Negotiation successful
0811 06:31:57.639598 (+ 671us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14322,"thread_start_us":14119,"threads_started":1}
I20260811 06:31:57.642617 15566 heartbeater.cc:347] Connected to a master server at 127.28.239.254:33707
I20260811 06:31:57.642983 15566 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.28.239.195@KRBTEST.COM; defaulting to no policy
I20260811 06:31:57.643802 15566 heartbeater.cc:511] Master 127.28.239.254:33707 requested a full tablet report, sending...
I20260811 06:31:57.645440 15035 ts_manager.cc:194] Registered new tserver with Master: f23311a2e25846e399721d66af185492 (127.28.239.194:42931)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.239.195@KRBTEST.COM" created.
I20260811 06:31:57.646440 15035 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.239.194@KRBTEST.COM'} at 127.28.239.194:50679
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.239.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.195.keytab.
Entry for principal kudu/127.28.239.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.195.keytab.
I20260811 06:31:57.682540 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.28.239.212:35765/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.195:0
--local_ip_for_outbound_sockets=127.28.239.195
--webserver_interface=127.28.239.195
--webserver_port=0
--tserver_master_addrs=127.28.239.254:33707
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.195.keytab
--principal=kudu/127.28.239.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:39397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:31:58.050805 15573 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:31:58.051185 15573 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:31:58.051378 15573 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:31:58.059808 15573 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:31:58.059989 15573 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:31:58.060199 15573 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.195
I20260811 06:31:58.071679 15573 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:39397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.28.239.212:35765/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.195.keytab
--principal=kudu/127.28.239.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.239.195
--webserver_port=0
--tserver_master_addrs=127.28.239.254:33707
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.195
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:31:58.073324 15573 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:31:58.074836 15573 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:31:58.089920 15579 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:31:58.089941 15580 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:31:58.090337 15584 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:31:58.091228 15573 server_base.cc:1034] running on GCE node
I20260811 06:31:58.092015 15573 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:31:58.093089 15573 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:31:58.094324 15573 hybrid_clock.cc:648] HybridClock initialized: now 1786429918094277 us; error 37 us; skew 500 ppm
I20260811 06:31:58.094699 15573 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:31:58 dist-test-slave-gkw7 krb5kdc[14272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429918, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:31:58.098095 15573 init.cc:377] Logged in from keytab as kudu/127.28.239.195@KRBTEST.COM (short username kudu)
I20260811 06:31:58.099762 15573 webserver.cc:460] Webserver started at http://127.28.239.195:40301/ using document root <none> and password file <none>
I20260811 06:31:58.100468 15573 fs_manager.cc:359] Metadata directory not provided
I20260811 06:31:58.100621 15573 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:31:58.100957 15573 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 06:31:58 dist-test-slave-gkw7 krb5kdc[14272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429918, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.195@KRBTEST.COM for HTTP/127.28.239.212@KRBTEST.COM
I20260811 06:31:58.116858 15573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/minicluster-data/ts-2/data/instance:
uuid: "01f6118838fa419dac6684ce9bf2a240"
format_stamp: "Formatted at 2026-08-11 06:31:58 on dist-test-slave-gkw7"
server_key: "b2bc40596f43a4c4fb897fd6adcd04a1"
server_key_iv: "823f99a45ee6e17c3219c3e56243b6dd"
server_key_version: "kuduclusterkey@0"
I20260811 06:31:58.117843 15573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/minicluster-data/ts-2/wal/instance:
uuid: "01f6118838fa419dac6684ce9bf2a240"
format_stamp: "Formatted at 2026-08-11 06:31:58 on dist-test-slave-gkw7"
server_key: "b2bc40596f43a4c4fb897fd6adcd04a1"
server_key_iv: "823f99a45ee6e17c3219c3e56243b6dd"
server_key_version: "kuduclusterkey@0"
I20260811 06:31:58.123498 15573 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 06:31:58.138545 15593 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:31:58.140507 15573 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 06:31:58.140726 15573 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/minicluster-data/ts-2/wal
uuid: "01f6118838fa419dac6684ce9bf2a240"
format_stamp: "Formatted at 2026-08-11 06:31:58 on dist-test-slave-gkw7"
server_key: "b2bc40596f43a4c4fb897fd6adcd04a1"
server_key_iv: "823f99a45ee6e17c3219c3e56243b6dd"
server_key_version: "kuduclusterkey@0"
I20260811 06:31:58.141052 15573 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:31:58.157043 15573 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:31:58.161837 15573 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:31:58.162194 15573 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:31:58.163280 15573 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:31:58.164866 15573 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:31:58.164984 15573 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:31:58.165150 15573 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:31:58.165236 15573 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:31:58.204629 15573 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.195:43335
I20260811 06:31:58.204671 15756 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.195:43335 every 8 connection(s)
I20260811 06:31:58.206384 15573 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
I20260811 06:31:58.216468 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 15573
I20260811 06:31:58.216665 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/minicluster-data/ts-2/wal/instance
I20260811 06:31:58.217196 29631 external_mini_cluster.cc:1428] Setting key 98966a7345698eeed1a355fc87e72e8b
Aug 11 06:31:58 dist-test-slave-gkw7 krb5kdc[14272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429918, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.195@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:31:58.236003 15764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:31:58.212871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.195:46611 (local address 127.28.239.254:33707)
0811 06:31:58.218180 (+ 5309us) server_negotiation.cc:206] Beginning negotiation
0811 06:31:58.218186 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:31:58.218206 (+ 20us) server_negotiation.cc:407] Connection header received
0811 06:31:58.218261 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:31:58.218268 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:31:58.218375 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:31:58.218506 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:31:58.220382 (+ 1876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:31:58.221496 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:31:58.224036 (+ 2540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:31:58.224760 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:31:58.224901 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:31:58.228888 (+ 3987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:31:58.228900 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:31:58.228905 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:31:58.228941 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:31:58.232233 (+ 3292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:31:58.233235 (+ 1002us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:31:58.233243 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:31:58.233247 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:31:58.233408 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:31:58.234055 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:31:58.234059 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:31:58.234061 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:31:58.234369 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:31:58.234552 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0811 06:31:58.235320 (+ 768us) server_negotiation.cc:299] Negotiation successful
0811 06:31:58.235554 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5169,"thread_start_us":4798,"threads_started":1}
I20260811 06:31:58.237084 15763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:31:58.210548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:33707 (local address 127.28.239.195:46611)
0811 06:31:58.211490 (+ 942us) negotiation.cc:107] Waiting for socket to connect
0811 06:31:58.211538 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 06:31:58.213959 (+ 2421us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:31:58.218808 (+ 4849us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:31:58.218820 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:31:58.219398 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:31:58.220210 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:31:58.220234 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:31:58.221888 (+ 1654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:31:58.221895 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:31:58.223888 (+ 1993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:31:58.223911 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:31:58.224999 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:31:58.225008 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:31:58.225310 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:31:58.226124 (+ 814us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:31:58.226158 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:31:58.228631 (+ 2473us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:31:58.232391 (+ 3760us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:31:58.232402 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:31:58.232418 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:31:58.233098 (+ 680us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:31:58.233578 (+ 480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:31:58.233584 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:31:58.233587 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:31:58.233891 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:31:58.234544 (+ 653us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:31:58.234552 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:31:58.234918 (+ 366us) client_negotiation.cc:769] Sending connection context
0811 06:31:58.235215 (+ 297us) client_negotiation.cc:240] Negotiation successful
0811 06:31:58.235833 (+ 618us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":638,"thread_start_us":362,"threads_started":1}
I20260811 06:31:58.238511 15758 heartbeater.cc:347] Connected to a master server at 127.28.239.254:33707
I20260811 06:31:58.238853 15758 heartbeater.cc:464] Registering TS with master...
I20260811 06:31:58.239689 15758 heartbeater.cc:511] Master 127.28.239.254:33707 requested a full tablet report, sending...
I20260811 06:31:58.241376 15035 ts_manager.cc:194] Registered new tserver with Master: 01f6118838fa419dac6684ce9bf2a240 (127.28.239.195:43335)
I20260811 06:31:58.242498 15035 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.239.195@KRBTEST.COM'} at 127.28.239.195:46611
I20260811 06:31:58.246619 29631 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 06:31:58.266171 15764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:31:58.251897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58620 (local address 127.28.239.254:33707)
0811 06:31:58.252102 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 06:31:58.252107 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:31:58.252709 (+ 602us) server_negotiation.cc:407] Connection header received
0811 06:31:58.252877 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:31:58.252881 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:31:58.252959 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:31:58.253043 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:31:58.254819 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:31:58.255579 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:31:58.257200 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:31:58.257700 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:31:58.257798 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:31:58.260326 (+ 2528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:31:58.260332 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:31:58.260336 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:31:58.260377 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:31:58.263440 (+ 3063us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:31:58.264109 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:31:58.264113 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:31:58.264115 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:31:58.264247 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:31:58.264714 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:31:58.264718 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:31:58.264720 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:31:58.265026 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:31:58.265285 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0811 06:31:58.265738 (+ 453us) server_negotiation.cc:299] Negotiation successful
0811 06:31:58.265915 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 06:31:58.270330 15035 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58620:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:31:58.277774 15035 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:31:58.305372 15784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:31:58.293775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60136 (local address 127.28.239.193:43153)
0811 06:31:58.295552 (+ 1777us) server_negotiation.cc:206] Beginning negotiation
0811 06:31:58.295563 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 06:31:58.295611 (+ 48us) server_negotiation.cc:407] Connection header received
0811 06:31:58.295731 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:31:58.295739 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:31:58.295991 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:31:58.297118 (+ 1127us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:31:58.297369 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:31:58.299156 (+ 1787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:31:58.302900 (+ 3744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:31:58.303980 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:31:58.304230 (+ 250us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:31:58.304440 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0811 06:31:58.304748 (+ 308us) server_negotiation.cc:299] Negotiation successful
0811 06:31:58.304940 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1581,"thread_start_us":1425,"threads_started":1}
I20260811 06:31:58.305457 15781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:31:58.291885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.193:43153 (local address 127.0.0.1:60136)
0811 06:31:58.293699 (+ 1814us) negotiation.cc:107] Waiting for socket to connect
0811 06:31:58.293719 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 06:31:58.293839 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:31:58.296250 (+ 2411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:31:58.296255 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:31:58.296279 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:31:58.296867 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:31:58.296879 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:31:58.299305 (+ 2426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:31:58.299312 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:31:58.302740 (+ 3428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:31:58.302753 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:31:58.304225 (+ 1472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:31:58.304232 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:31:58.304438 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:31:58.304517 (+ 79us) client_negotiation.cc:769] Sending connection context
0811 06:31:58.304684 (+ 167us) client_negotiation.cc:240] Negotiation successful
0811 06:31:58.304940 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1664,"mutex_wait_us":345,"thread_start_us":1470,"threads_started":1}
I20260811 06:31:58.305966 15780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:31:58.291688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.195:43335 (local address 127.0.0.1:54338)
0811 06:31:58.292644 (+ 956us) negotiation.cc:107] Waiting for socket to connect
0811 06:31:58.292711 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0811 06:31:58.292939 (+ 228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:31:58.296729 (+ 3790us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:31:58.296735 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:31:58.296764 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:31:58.297303 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:31:58.297317 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:31:58.299195 (+ 1878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:31:58.299207 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:31:58.302767 (+ 3560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:31:58.302785 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:31:58.304870 (+ 2085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:31:58.304877 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:31:58.305101 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:31:58.305130 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 06:31:58.305193 (+ 63us) client_negotiation.cc:240] Negotiation successful
0811 06:31:58.305454 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":593,"mutex_wait_us":635,"thread_start_us":417,"threads_started":1}
I20260811 06:31:58.306519 15783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:31:58.293496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35532 (local address 127.28.239.194:42931)
0811 06:31:58.294398 (+ 902us) server_negotiation.cc:206] Beginning negotiation
0811 06:31:58.294405 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:31:58.294869 (+ 464us) server_negotiation.cc:407] Connection header received
0811 06:31:58.295132 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:31:58.295144 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:31:58.295459 (+ 315us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:31:58.296893 (+ 1434us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:31:58.297166 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:31:58.299034 (+ 1868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:31:58.302532 (+ 3498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:31:58.304218 (+ 1686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:31:58.304398 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:31:58.304572 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0811 06:31:58.305890 (+ 1318us) server_negotiation.cc:299] Negotiation successful
0811 06:31:58.306111 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":743,"thread_start_us":574,"threads_started":1}
I20260811 06:31:58.306972 15782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:31:58.293211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.194:42931 (local address 127.0.0.1:35532)
0811 06:31:58.294744 (+ 1533us) negotiation.cc:107] Waiting for socket to connect
0811 06:31:58.294758 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 06:31:58.294912 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:31:58.295746 (+ 834us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:31:58.295755 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:31:58.295788 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:31:58.296618 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:31:58.296640 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:31:58.299359 (+ 2719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:31:58.299363 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:31:58.302350 (+ 2987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:31:58.302367 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:31:58.304762 (+ 2395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:31:58.304772 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:31:58.305007 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:31:58.305101 (+ 94us) client_negotiation.cc:769] Sending connection context
0811 06:31:58.305198 (+ 97us) client_negotiation.cc:240] Negotiation successful
0811 06:31:58.306402 (+ 1204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1441,"mutex_wait_us":1429,"thread_start_us":1347,"threads_started":1}
I20260811 06:31:58.308188 15778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:31:58.291671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54338 (local address 127.28.239.195:43335)
0811 06:31:58.296031 (+ 4360us) server_negotiation.cc:206] Beginning negotiation
0811 06:31:58.296038 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:31:58.296098 (+ 60us) server_negotiation.cc:407] Connection header received
0811 06:31:58.296208 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:31:58.296220 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:31:58.296466 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:31:58.297503 (+ 1037us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:31:58.297888 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:31:58.298992 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:31:58.302970 (+ 3978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:31:58.304603 (+ 1633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:31:58.307284 (+ 2681us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:31:58.307532 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0811 06:31:58.307617 (+ 85us) server_negotiation.cc:299] Negotiation successful
0811 06:31:58.307787 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4194,"thread_start_us":4035,"threads_started":1}
I20260811 06:31:58.311131 15478 tablet_service.cc:1467] Processing CreateTablet for tablet a9e5095064a44caf99d48060dca9fce5 (DEFAULT_TABLE table=test-table [id=9d06d4e6321e46808d49456a0e1ec155]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:31:58.310905 15253 tablet_service.cc:1467] Processing CreateTablet for tablet a9e5095064a44caf99d48060dca9fce5 (DEFAULT_TABLE table=test-table [id=9d06d4e6321e46808d49456a0e1ec155]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:31:58.311944 15662 tablet_service.cc:1467] Processing CreateTablet for tablet a9e5095064a44caf99d48060dca9fce5 (DEFAULT_TABLE table=test-table [id=9d06d4e6321e46808d49456a0e1ec155]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:31:58.312656 15478 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a9e5095064a44caf99d48060dca9fce5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:31:58.312656 15253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a9e5095064a44caf99d48060dca9fce5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:31:58.313310 15662 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a9e5095064a44caf99d48060dca9fce5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:31:58.326048 15787 tablet_bootstrap.cc:492] T a9e5095064a44caf99d48060dca9fce5 P 01f6118838fa419dac6684ce9bf2a240: Bootstrap starting.
I20260811 06:31:58.329610 15788 tablet_bootstrap.cc:492] T a9e5095064a44caf99d48060dca9fce5 P f23311a2e25846e399721d66af185492: Bootstrap starting.
I20260811 06:31:58.331668 15787 tablet_bootstrap.cc:654] T a9e5095064a44caf99d48060dca9fce5 P 01f6118838fa419dac6684ce9bf2a240: Neither blocks nor log segments found. Creating new log.
I20260811 06:31:58.333312 15787 log.cc:824] T a9e5095064a44caf99d48060dca9fce5 P 01f6118838fa419dac6684ce9bf2a240: Log is configured to *not* fsync() on all Append() calls
I20260811 06:31:58.333902 15788 tablet_bootstrap.cc:654] T a9e5095064a44caf99d48060dca9fce5 P f23311a2e25846e399721d66af185492: Neither blocks nor log segments found. Creating new log.
I20260811 06:31:58.334362 15790 tablet_bootstrap.cc:492] T a9e5095064a44caf99d48060dca9fce5 P f4ac3d7319bc43769f4e2cc48adee3ff: Bootstrap starting.
I20260811 06:31:58.335530 15788 log.cc:824] T a9e5095064a44caf99d48060dca9fce5 P f23311a2e25846e399721d66af185492: Log is configured to *not* fsync() on all Append() calls
I20260811 06:31:58.338315 15790 tablet_bootstrap.cc:654] T a9e5095064a44caf99d48060dca9fce5 P f4ac3d7319bc43769f4e2cc48adee3ff: Neither blocks nor log segments found. Creating new log.
I20260811 06:31:58.339211 15787 tablet_bootstrap.cc:492] T a9e5095064a44caf99d48060dca9fce5 P 01f6118838fa419dac6684ce9bf2a240: No bootstrap required, opened a new log
I20260811 06:31:58.339581 15790 log.cc:824] T a9e5095064a44caf99d48060dca9fce5 P f4ac3d7319bc43769f4e2cc48adee3ff: Log is configured to *not* fsync() on all Append() calls
I20260811 06:31:58.339592 15787 ts_tablet_manager.cc:1397] T a9e5095064a44caf99d48060dca9fce5 P 01f6118838fa419dac6684ce9bf2a240: Time spent bootstrapping tablet: real 0.014s user 0.008s sys 0.001s
I20260811 06:31:58.343115 15790 tablet_bootstrap.cc:492] T a9e5095064a44caf99d48060dca9fce5 P f4ac3d7319bc43769f4e2cc48adee3ff: No bootstrap required, opened a new log
I20260811 06:31:58.343415 15790 ts_tablet_manager.cc:1397] T a9e5095064a44caf99d48060dca9fce5 P f4ac3d7319bc43769f4e2cc48adee3ff: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.007s
I20260811 06:31:58.344466 15788 tablet_bootstrap.cc:492] T a9e5095064a44caf99d48060dca9fce5 P f23311a2e25846e399721d66af185492: No bootstrap required, opened a new log
I20260811 06:31:58.344769 15788 ts_tablet_manager.cc:1397] T a9e5095064a44caf99d48060dca9fce5 P f23311a2e25846e399721d66af185492: Time spent bootstrapping tablet: real 0.015s user 0.003s sys 0.010s
I20260811 06:31:58.352553 15790 raft_consensus.cc:348] T a9e5095064a44caf99d48060dca9fce5 P f4ac3d7319bc43769f4e2cc48adee3ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01f6118838fa419dac6684ce9bf2a240" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 43335 } } peers { permanent_uuid: "f23311a2e25846e399721d66af185492" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 42931 } } peers { permanent_uuid: "f4ac3d7319bc43769f4e2cc48adee3ff" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 43153 } }
I20260811 06:31:58.352564 15787 raft_consensus.cc:348] T a9e5095064a44caf99d48060dca9fce5 P 01f6118838fa419dac6684ce9bf2a240 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01f6118838fa419dac6684ce9bf2a240" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 43335 } } peers { permanent_uuid: "f23311a2e25846e399721d66af185492" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 42931 } } peers { permanent_uuid: "f4ac3d7319bc43769f4e2cc48adee3ff" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 43153 } }
I20260811 06:31:58.352985 15790 raft_consensus.cc:374] T a9e5095064a44caf99d48060dca9fce5 P f4ac3d7319bc43769f4e2cc48adee3ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:31:58.353024 15787 raft_consensus.cc:374] T a9e5095064a44caf99d48060dca9fce5 P 01f6118838fa419dac6684ce9bf2a240 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:31:58.353123 15790 raft_consensus.cc:729] T a9e5095064a44caf99d48060dca9fce5 P f4ac3d7319bc43769f4e2cc48adee3ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4ac3d7319bc43769f4e2cc48adee3ff, State: Initialized, Role: FOLLOWER
I20260811 06:31:58.353156 15787 raft_consensus.cc:729] T a9e5095064a44caf99d48060dca9fce5 P 01f6118838fa419dac6684ce9bf2a240 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01f6118838fa419dac6684ce9bf2a240, State: Initialized, Role: FOLLOWER
I20260811 06:31:58.353847 15790 consensus_queue.cc:260] T a9e5095064a44caf99d48060dca9fce5 P f4ac3d7319bc43769f4e2cc48adee3ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01f6118838fa419dac6684ce9bf2a240" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 43335 } } peers { permanent_uuid: "f23311a2e25846e399721d66af185492" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 42931 } } peers { permanent_uuid: "f4ac3d7319bc43769f4e2cc48adee3ff" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 43153 } }
I20260811 06:31:58.353950 15787 consensus_queue.cc:260] T a9e5095064a44caf99d48060dca9fce5 P 01f6118838fa419dac6684ce9bf2a240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01f6118838fa419dac6684ce9bf2a240" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 43335 } } peers { permanent_uuid: "f23311a2e25846e399721d66af185492" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 42931 } } peers { permanent_uuid: "f4ac3d7319bc43769f4e2cc48adee3ff" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 43153 } }
I20260811 06:31:58.357016 15790 ts_tablet_manager.cc:1428] T a9e5095064a44caf99d48060dca9fce5 P f4ac3d7319bc43769f4e2cc48adee3ff: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260811 06:31:58.357736 15788 raft_consensus.cc:348] T a9e5095064a44caf99d48060dca9fce5 P f23311a2e25846e399721d66af185492 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01f6118838fa419dac6684ce9bf2a240" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 43335 } } peers { permanent_uuid: "f23311a2e25846e399721d66af185492" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 42931 } } peers { permanent_uuid: "f4ac3d7319bc43769f4e2cc48adee3ff" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 43153 } }
I20260811 06:31:58.358074 15758 heartbeater.cc:503] Master 127.28.239.254:33707 was elected leader, sending a full tablet report...
I20260811 06:31:58.358188 15788 raft_consensus.cc:374] T a9e5095064a44caf99d48060dca9fce5 P f23311a2e25846e399721d66af185492 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:31:58.358301 15788 raft_consensus.cc:729] T a9e5095064a44caf99d48060dca9fce5 P f23311a2e25846e399721d66af185492 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f23311a2e25846e399721d66af185492, State: Initialized, Role: FOLLOWER
I20260811 06:31:58.358841 15787 ts_tablet_manager.cc:1428] T a9e5095064a44caf99d48060dca9fce5 P 01f6118838fa419dac6684ce9bf2a240: Time spent starting tablet: real 0.019s user 0.016s sys 0.003s
I20260811 06:31:58.358989 15788 consensus_queue.cc:260] T a9e5095064a44caf99d48060dca9fce5 P f23311a2e25846e399721d66af185492 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01f6118838fa419dac6684ce9bf2a240" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 43335 } } peers { permanent_uuid: "f23311a2e25846e399721d66af185492" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 42931 } } peers { permanent_uuid: "f4ac3d7319bc43769f4e2cc48adee3ff" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 43153 } }
I20260811 06:31:58.361366 15566 heartbeater.cc:503] Master 127.28.239.254:33707 was elected leader, sending a full tablet report...
I20260811 06:31:58.362124 15788 ts_tablet_manager.cc:1428] T a9e5095064a44caf99d48060dca9fce5 P f23311a2e25846e399721d66af185492: Time spent starting tablet: real 0.017s user 0.013s sys 0.006s
W20260811 06:31:58.458184 15760 tablet.cc:2369] T a9e5095064a44caf99d48060dca9fce5 P 01f6118838fa419dac6684ce9bf2a240: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:31:58.479511 15795 raft_consensus.cc:482] T a9e5095064a44caf99d48060dca9fce5 P f4ac3d7319bc43769f4e2cc48adee3ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:31:58.479877 15795 raft_consensus.cc:504] T a9e5095064a44caf99d48060dca9fce5 P f4ac3d7319bc43769f4e2cc48adee3ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01f6118838fa419dac6684ce9bf2a240" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 43335 } } peers { permanent_uuid: "f23311a2e25846e399721d66af185492" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 42931 } } peers { permanent_uuid: "f4ac3d7319bc43769f4e2cc48adee3ff" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 43153 } }
I20260811 06:31:58.481801 15795 leader_election.cc:288] T a9e5095064a44caf99d48060dca9fce5 P f4ac3d7319bc43769f4e2cc48adee3ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01f6118838fa419dac6684ce9bf2a240 (127.28.239.195:43335), f23311a2e25846e399721d66af185492 (127.28.239.194:42931)
I20260811 06:31:58.489427 15778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:31:58.482272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.193:33307 (local address 127.28.239.195:43335)
0811 06:31:58.482464 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 06:31:58.482468 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:31:58.483197 (+ 729us) server_negotiation.cc:407] Connection header received
0811 06:31:58.483520 (+ 323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:31:58.483528 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:31:58.483614 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:31:58.483688 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:31:58.484611 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:31:58.485497 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:31:58.487303 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:31:58.488085 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:31:58.488275 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:31:58.488357 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 06:31:58.489014 (+ 657us) server_negotiation.cc:299] Negotiation successful
0811 06:31:58.489175 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 06:31:58.489503 15800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:31:58.482325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.195:43335 (local address 127.28.239.193:33307)
0811 06:31:58.482995 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0811 06:31:58.483012 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 06:31:58.483327 (+ 315us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:31:58.483903 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:31:58.483909 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:31:58.483934 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:31:58.484428 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:31:58.484442 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:31:58.485701 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:31:58.485707 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:31:58.487123 (+ 1416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:31:58.487136 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:31:58.488658 (+ 1522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:31:58.488665 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:31:58.488846 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:31:58.488871 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 06:31:58.488984 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 06:31:58.489118 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"thread_start_us":361,"threads_started":1}
I20260811 06:31:58.490298 15783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:31:58.483352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.193:36089 (local address 127.28.239.194:42931)
0811 06:31:58.483954 (+ 602us) server_negotiation.cc:206] Beginning negotiation
0811 06:31:58.483962 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:31:58.484605 (+ 643us) server_negotiation.cc:407] Connection header received
0811 06:31:58.484843 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:31:58.484847 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:31:58.484919 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:31:58.485019 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:31:58.485604 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:31:58.486312 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:31:58.488070 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:31:58.489071 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:31:58.489236 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:31:58.489303 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 06:31:58.489952 (+ 649us) server_negotiation.cc:299] Negotiation successful
0811 06:31:58.490107 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":441}
I20260811 06:31:58.490574 15691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9e5095064a44caf99d48060dca9fce5" candidate_uuid: "f4ac3d7319bc43769f4e2cc48adee3ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01f6118838fa419dac6684ce9bf2a240" is_pre_election: true
I20260811 06:31:58.490898 15801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:31:58.483149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.194:42931 (local address 127.28.239.193:36089)
0811 06:31:58.484498 (+ 1349us) negotiation.cc:107] Waiting for socket to connect
0811 06:31:58.484518 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 06:31:58.484695 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:31:58.485037 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:31:58.485041 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:31:58.485064 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:31:58.485419 (+ 355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:31:58.485428 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:31:58.486451 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:31:58.486456 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:31:58.487910 (+ 1454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:31:58.487923 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:31:58.489477 (+ 1554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:31:58.489490 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:31:58.489735 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:31:58.489782 (+ 47us) client_negotiation.cc:769] Sending connection context
0811 06:31:58.490504 (+ 722us) client_negotiation.cc:240] Negotiation successful
0811 06:31:58.490621 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1217,"thread_start_us":999,"threads_started":1}
I20260811 06:31:58.491148 15691 raft_consensus.cc:2454] T a9e5095064a44caf99d48060dca9fce5 P 01f6118838fa419dac6684ce9bf2a240 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f4ac3d7319bc43769f4e2cc48adee3ff in term 0.
I20260811 06:31:58.491866 15190 leader_election.cc:302] T a9e5095064a44caf99d48060dca9fce5 P f4ac3d7319bc43769f4e2cc48adee3ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01f6118838fa419dac6684ce9bf2a240, f4ac3d7319bc43769f4e2cc48adee3ff; no voters:
I20260811 06:31:58.491993 15510 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9e5095064a44caf99d48060dca9fce5" candidate_uuid: "f4ac3d7319bc43769f4e2cc48adee3ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f23311a2e25846e399721d66af185492" is_pre_election: true
I20260811 06:31:58.492446 15795 raft_consensus.cc:2781] T a9e5095064a44caf99d48060dca9fce5 P f4ac3d7319bc43769f4e2cc48adee3ff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:31:58.492596 15795 raft_consensus.cc:482] T a9e5095064a44caf99d48060dca9fce5 P f4ac3d7319bc43769f4e2cc48adee3ff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:31:58.492573 15510 raft_consensus.cc:2454] T a9e5095064a44caf99d48060dca9fce5 P f23311a2e25846e399721d66af185492 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f4ac3d7319bc43769f4e2cc48adee3ff in term 0.
I20260811 06:31:58.492697 15795 raft_consensus.cc:3037] T a9e5095064a44caf99d48060dca9fce5 P f4ac3d7319bc43769f4e2cc48adee3ff [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:31:58.495177 15795 raft_consensus.cc:504] T a9e5095064a44caf99d48060dca9fce5 P f4ac3d7319bc43769f4e2cc48adee3ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01f6118838fa419dac6684ce9bf2a240" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 43335 } } peers { permanent_uuid: "f23311a2e25846e399721d66af185492" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 42931 } } peers { permanent_uuid: "f4ac3d7319bc43769f4e2cc48adee3ff" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 43153 } }
I20260811 06:31:58.495814 15795 leader_election.cc:288] T a9e5095064a44caf99d48060dca9fce5 P f4ac3d7319bc43769f4e2cc48adee3ff [CANDIDATE]: Term 1 election: Requested vote from peers 01f6118838fa419dac6684ce9bf2a240 (127.28.239.195:43335), f23311a2e25846e399721d66af185492 (127.28.239.194:42931)
I20260811 06:31:58.496277 15691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9e5095064a44caf99d48060dca9fce5" candidate_uuid: "f4ac3d7319bc43769f4e2cc48adee3ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01f6118838fa419dac6684ce9bf2a240"
I20260811 06:31:58.496310 15510 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9e5095064a44caf99d48060dca9fce5" candidate_uuid: "f4ac3d7319bc43769f4e2cc48adee3ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f23311a2e25846e399721d66af185492"
I20260811 06:31:58.496460 15691 raft_consensus.cc:3037] T a9e5095064a44caf99d48060dca9fce5 P 01f6118838fa419dac6684ce9bf2a240 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:31:58.496522 15510 raft_consensus.cc:3037] T a9e5095064a44caf99d48060dca9fce5 P f23311a2e25846e399721d66af185492 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:31:58.498842 15691 raft_consensus.cc:2454] T a9e5095064a44caf99d48060dca9fce5 P 01f6118838fa419dac6684ce9bf2a240 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4ac3d7319bc43769f4e2cc48adee3ff in term 1.
I20260811 06:31:58.498898 15510 raft_consensus.cc:2454] T a9e5095064a44caf99d48060dca9fce5 P f23311a2e25846e399721d66af185492 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4ac3d7319bc43769f4e2cc48adee3ff in term 1.
I20260811 06:31:58.499264 15190 leader_election.cc:302] T a9e5095064a44caf99d48060dca9fce5 P f4ac3d7319bc43769f4e2cc48adee3ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01f6118838fa419dac6684ce9bf2a240, f4ac3d7319bc43769f4e2cc48adee3ff; no voters:
I20260811 06:31:58.499574 15795 raft_consensus.cc:2781] T a9e5095064a44caf99d48060dca9fce5 P f4ac3d7319bc43769f4e2cc48adee3ff [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:31:58.500023 15795 raft_consensus.cc:686] T a9e5095064a44caf99d48060dca9fce5 P f4ac3d7319bc43769f4e2cc48adee3ff [term 1 LEADER]: Becoming Leader. State: Replica: f4ac3d7319bc43769f4e2cc48adee3ff, State: Running, Role: LEADER
I20260811 06:31:58.500614 15795 consensus_queue.cc:237] T a9e5095064a44caf99d48060dca9fce5 P f4ac3d7319bc43769f4e2cc48adee3ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01f6118838fa419dac6684ce9bf2a240" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 43335 } } peers { permanent_uuid: "f23311a2e25846e399721d66af185492" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 42931 } } peers { permanent_uuid: "f4ac3d7319bc43769f4e2cc48adee3ff" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 43153 } }
W20260811 06:31:58.501437 15359 tablet.cc:2369] T a9e5095064a44caf99d48060dca9fce5 P f4ac3d7319bc43769f4e2cc48adee3ff: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:31:58.505927 15035 catalog_manager.cc:5495] T a9e5095064a44caf99d48060dca9fce5 P f4ac3d7319bc43769f4e2cc48adee3ff reported cstate change: term changed from 0 to 1, leader changed from <none> to f4ac3d7319bc43769f4e2cc48adee3ff (127.28.239.193). New cstate: current_term: 1 leader_uuid: "f4ac3d7319bc43769f4e2cc48adee3ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01f6118838fa419dac6684ce9bf2a240" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 43335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f23311a2e25846e399721d66af185492" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 42931 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f4ac3d7319bc43769f4e2cc48adee3ff" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 43153 } health_report { overall_health: HEALTHY } } }
I20260811 06:31:58.549500 15784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:31:58.543676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60150 (local address 127.28.239.193:43153)
0811 06:31:58.543940 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0811 06:31:58.543947 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:31:58.543968 (+ 21us) server_negotiation.cc:407] Connection header received
0811 06:31:58.544127 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:31:58.544132 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:31:58.544217 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:31:58.544399 (+ 182us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:31:58.544784 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:31:58.545610 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:31:58.547032 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:31:58.547454 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:31:58.547615 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:31:58.548133 (+ 518us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:31:58.548764 (+ 631us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:31:58.548931 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0811 06:31:58.548991 (+ 60us) server_negotiation.cc:299] Negotiation successful
0811 06:31:58.549076 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 06:31:58.558398 15510 raft_consensus.cc:1261] T a9e5095064a44caf99d48060dca9fce5 P f23311a2e25846e399721d66af185492 [term 1 FOLLOWER]: Refusing update from remote peer f4ac3d7319bc43769f4e2cc48adee3ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:31:58.558398 15691 raft_consensus.cc:1261] T a9e5095064a44caf99d48060dca9fce5 P 01f6118838fa419dac6684ce9bf2a240 [term 1 FOLLOWER]: Refusing update from remote peer f4ac3d7319bc43769f4e2cc48adee3ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:31:58.559454 15803 consensus_queue.cc:1035] T a9e5095064a44caf99d48060dca9fce5 P f4ac3d7319bc43769f4e2cc48adee3ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "01f6118838fa419dac6684ce9bf2a240" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 43335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:31:58.559892 15795 consensus_queue.cc:1035] T a9e5095064a44caf99d48060dca9fce5 P f4ac3d7319bc43769f4e2cc48adee3ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "f23311a2e25846e399721d66af185492" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 42931 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:31:58.572973 15808 mvcc.cc:205] Tried to move back new op lower bound from 7317216946403639296 to 7317216946183290880. Current Snapshot: MvccSnapshot[applied={T|T < 7317216946403639296}]
I20260811 06:31:58.574585 15810 mvcc.cc:205] Tried to move back new op lower bound from 7317216946403639296 to 7317216946183290880. Current Snapshot: MvccSnapshot[applied={T|T < 7317216946403639296}]
I20260811 06:31:58.587864 15035 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58620:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:31:58.588137 15035 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58620:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:31:58.591841 15035 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e5486c2f2a504308aa5f91c8758b2411: Sending DeleteTablet for 3 replicas of tablet a9e5095064a44caf99d48060dca9fce5
I20260811 06:31:58.593017 15662 tablet_service.cc:1514] Processing DeleteTablet for tablet a9e5095064a44caf99d48060dca9fce5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:31:58 UTC) from {username='kudu', principal='kudu/127.28.239.254@KRBTEST.COM'} at 127.0.0.1:54338
I20260811 06:31:58.593163 15478 tablet_service.cc:1514] Processing DeleteTablet for tablet a9e5095064a44caf99d48060dca9fce5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:31:58 UTC) from {username='kudu', principal='kudu/127.28.239.254@KRBTEST.COM'} at 127.0.0.1:35532
I20260811 06:31:58.593461 15253 tablet_service.cc:1514] Processing DeleteTablet for tablet a9e5095064a44caf99d48060dca9fce5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:31:58 UTC) from {username='kudu', principal='kudu/127.28.239.254@KRBTEST.COM'} at 127.0.0.1:60136
I20260811 06:31:58.599479 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 15153
I20260811 06:31:58.618482 15822 tablet_replica.cc:330] T a9e5095064a44caf99d48060dca9fce5 P f4ac3d7319bc43769f4e2cc48adee3ff: stopping tablet replica
I20260811 06:31:58.619204 15822 raft_consensus.cc:2229] T a9e5095064a44caf99d48060dca9fce5 P f4ac3d7319bc43769f4e2cc48adee3ff [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:31:58.620114 15822 raft_consensus.cc:2258] T a9e5095064a44caf99d48060dca9fce5 P f4ac3d7319bc43769f4e2cc48adee3ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:31:58.622531 15822 ts_tablet_manager.cc:1905] T a9e5095064a44caf99d48060dca9fce5 P f4ac3d7319bc43769f4e2cc48adee3ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:31:58.626647 15821 tablet_replica.cc:330] T a9e5095064a44caf99d48060dca9fce5 P f23311a2e25846e399721d66af185492: stopping tablet replica
I20260811 06:31:58.627097 15821 raft_consensus.cc:2229] T a9e5095064a44caf99d48060dca9fce5 P f23311a2e25846e399721d66af185492 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:31:58.627696 15821 raft_consensus.cc:2258] T a9e5095064a44caf99d48060dca9fce5 P f23311a2e25846e399721d66af185492 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:31:58.631174 15822 ts_tablet_manager.cc:1918] T a9e5095064a44caf99d48060dca9fce5 P f4ac3d7319bc43769f4e2cc48adee3ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:31:58.631429 15822 log.cc:1196] T a9e5095064a44caf99d48060dca9fce5 P f4ac3d7319bc43769f4e2cc48adee3ff: Deleting WAL directory at /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/minicluster-data/ts-0/wal/wals/a9e5095064a44caf99d48060dca9fce5
I20260811 06:31:58.632122 15822 ts_tablet_manager.cc:1939] T a9e5095064a44caf99d48060dca9fce5 P f4ac3d7319bc43769f4e2cc48adee3ff: Deleting consensus metadata
I20260811 06:31:58.633911 15821 ts_tablet_manager.cc:1905] T a9e5095064a44caf99d48060dca9fce5 P f23311a2e25846e399721d66af185492: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:31:58.634325 15004 catalog_manager.cc:4841] TS f4ac3d7319bc43769f4e2cc48adee3ff (127.28.239.193:43153): tablet a9e5095064a44caf99d48060dca9fce5 (table test-table [id=9d06d4e6321e46808d49456a0e1ec155]) successfully deleted
I20260811 06:31:58.635520 15820 tablet_replica.cc:330] T a9e5095064a44caf99d48060dca9fce5 P 01f6118838fa419dac6684ce9bf2a240: stopping tablet replica
I20260811 06:31:58.635993 15820 raft_consensus.cc:2229] T a9e5095064a44caf99d48060dca9fce5 P 01f6118838fa419dac6684ce9bf2a240 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:31:58.636538 15820 raft_consensus.cc:2258] T a9e5095064a44caf99d48060dca9fce5 P 01f6118838fa419dac6684ce9bf2a240 [term 1 FOLLOWER]: Raft consensus is shut down!
Aug 11 06:31:58 dist-test-slave-gkw7 krb5kdc[14272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429914, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.239.193@KRBTEST.COM
I20260811 06:31:58.644651 15821 ts_tablet_manager.cc:1918] T a9e5095064a44caf99d48060dca9fce5 P f23311a2e25846e399721d66af185492: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:31:58.644825 15821 log.cc:1196] T a9e5095064a44caf99d48060dca9fce5 P f23311a2e25846e399721d66af185492: Deleting WAL directory at /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/minicluster-data/ts-1/wal/wals/a9e5095064a44caf99d48060dca9fce5
I20260811 06:31:58.641202 15820 ts_tablet_manager.cc:1905] T a9e5095064a44caf99d48060dca9fce5 P 01f6118838fa419dac6684ce9bf2a240: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:31:58.649359 15821 ts_tablet_manager.cc:1939] T a9e5095064a44caf99d48060dca9fce5 P f23311a2e25846e399721d66af185492: Deleting consensus metadata
I20260811 06:31:58.650316 15784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:31:58.601694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60154 (local address 127.28.239.193:43153)
0811 06:31:58.601902 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 06:31:58.601908 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:31:58.615223 (+ 13315us) server_negotiation.cc:407] Connection header received
0811 06:31:58.616301 (+ 1078us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:31:58.616310 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:31:58.616426 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:31:58.616522 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:31:58.628228 (+ 11706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:31:58.629604 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:31:58.634966 (+ 5362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:31:58.635723 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:31:58.635873 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:31:58.641688 (+ 5815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:31:58.641704 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:31:58.641724 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:31:58.641766 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:31:58.646062 (+ 4296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:31:58.647176 (+ 1114us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:31:58.647186 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:31:58.647189 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:31:58.647404 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:31:58.647930 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:31:58.647936 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:31:58.647939 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:31:58.648468 (+ 529us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:31:58.648780 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0811 06:31:58.649501 (+ 721us) server_negotiation.cc:299] Negotiation successful
0811 06:31:58.649795 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 06:31:58.651821 15005 catalog_manager.cc:4841] TS f23311a2e25846e399721d66af185492 (127.28.239.194:42931): tablet a9e5095064a44caf99d48060dca9fce5 (table test-table [id=9d06d4e6321e46808d49456a0e1ec155]) successfully deleted
I20260811 06:31:58.652459 15349 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:31:58.653436 15820 ts_tablet_manager.cc:1918] T a9e5095064a44caf99d48060dca9fce5 P 01f6118838fa419dac6684ce9bf2a240: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:31:58.653671 15820 log.cc:1196] T a9e5095064a44caf99d48060dca9fce5 P 01f6118838fa419dac6684ce9bf2a240: Deleting WAL directory at /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786429755927508-29631-0/minicluster-data/ts-2/wal/wals/a9e5095064a44caf99d48060dca9fce5
I20260811 06:31:58.654279 15820 ts_tablet_manager.cc:1939] T a9e5095064a44caf99d48060dca9fce5 P 01f6118838fa419dac6684ce9bf2a240: Deleting consensus metadata
I20260811 06:31:58.655988 15005 catalog_manager.cc:4841] TS 01f6118838fa419dac6684ce9bf2a240 (127.28.239.195:43335): tablet a9e5095064a44caf99d48060dca9fce5 (table test-table [id=9d06d4e6321e46808d49456a0e1ec155]) successfully deleted
I20260811 06:31:58.754776 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 15153
I20260811 06:31:58.783607 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 15369
Aug 11 06:31:58 dist-test-slave-gkw7 krb5kdc[14272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429914, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.239.194@KRBTEST.COM
I20260811 06:31:58.799803 15783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:31:58.784608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35548 (local address 127.28.239.194:42931)
0811 06:31:58.784852 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0811 06:31:58.784858 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:31:58.784879 (+ 21us) server_negotiation.cc:407] Connection header received
0811 06:31:58.784946 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:31:58.784950 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:31:58.785012 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:31:58.785177 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:31:58.786565 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:31:58.787353 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:31:58.788733 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:31:58.789160 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:31:58.789310 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:31:58.793780 (+ 4470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:31:58.793797 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:31:58.793814 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:31:58.793863 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:31:58.796801 (+ 2938us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:31:58.797520 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:31:58.797574 (+ 54us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:31:58.797578 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:31:58.797727 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:31:58.798144 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:31:58.798156 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:31:58.798158 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:31:58.798550 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:31:58.798874 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0811 06:31:58.799195 (+ 321us) server_negotiation.cc:299] Negotiation successful
0811 06:31:58.799421 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 06:31:58.800607 15560 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:31:58.892541 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 15369
I20260811 06:31:58.924314 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 15573
Aug 11 06:31:58 dist-test-slave-gkw7 krb5kdc[14272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429914, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.239.195@KRBTEST.COM
I20260811 06:31:58.940656 15778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:31:58.925258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54354 (local address 127.28.239.195:43335)
0811 06:31:58.925454 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 06:31:58.925462 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:31:58.925483 (+ 21us) server_negotiation.cc:407] Connection header received
0811 06:31:58.925733 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:31:58.925738 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:31:58.925821 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:31:58.925936 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:31:58.927347 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:31:58.928089 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:31:58.929413 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:31:58.929981 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:31:58.930122 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:31:58.934590 (+ 4468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:31:58.934612 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:31:58.934630 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:31:58.934673 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:31:58.937786 (+ 3113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:31:58.938475 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:31:58.938486 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:31:58.938489 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:31:58.938651 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:31:58.939038 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:31:58.939043 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:31:58.939047 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:31:58.939444 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:31:58.939770 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0811 06:31:58.940068 (+ 298us) server_negotiation.cc:299] Negotiation successful
0811 06:31:58.940304 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 06:31:58.941365 15751 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:31:59.031576 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 15573
I20260811 06:31:59.060398 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 14982
I20260811 06:31:59.061903 15078 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:31:59.175997 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 14982
I20260811 06:31:59.216233 29631 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 06:31:59.561736 29631 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 06:31:59.562026 29631 mini_ranger.cc:66] Stopping Ranger...
I20260811 06:31:59.917764 29631 mini_ranger.cc:68] Stopped Ranger
2026-08-11 06:31:59.918 UTC [14324] LOG: received smart shutdown request
2026-08-11 06:31:59.922 UTC [14324] LOG: background worker "logical replication launcher" (PID 14338) exited with exit code 1
2026-08-11 06:31:59.922 UTC [14331] LOG: shutting down
2026-08-11 06:32:00.108 UTC [14324] LOG: database system is shut down
2026-08-11T06:32:00Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (55097 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:32:00 dist-test-slave-gkw7 krb5kdc[15844](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:32:00 dist-test-slave-gkw7 krb5kdc[15844](info): set up 2 sockets
Aug 11 06:32:00 dist-test-slave-gkw7 krb5kdc[15844](info): commencing operation
krb5kdc: starting...
W20260811 06:32:01.662145 29631 mini_kdc.cc:121] Time spent starting KDC: real 1.376s user 0.000s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/krb5kdc/test-user.keytab.
Aug 11 06:32:01 dist-test-slave-gkw7 krb5kdc[15844](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429921, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:32:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:32:01Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.28.239.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.28.239.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.28.239.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/krb5kdc/rangeradmin_127.28.239.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.28.239.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/krb5kdc/rangeradmin_127.28.239.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.28.239.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.28.239.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.28.239.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/krb5kdc/rangerlookup_127.28.239.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.28.239.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/krb5kdc/rangerlookup_127.28.239.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.28.239.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.28.239.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.28.239.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/krb5kdc/HTTP_127.28.239.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.28.239.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/krb5kdc/HTTP_127.28.239.212@KRBTEST.COM.keytab.
I20260811 06:32:02.098223 29631 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskJveEHt/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/postgres -l logfile start
2026-08-11 06:32:04.792 UTC [15894] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 06:32:04.792 UTC [15894] LOG: listening on IPv4 address "127.28.239.212", port 54857
2026-08-11 06:32:04.796 UTC [15894] LOG: listening on Unix socket "/tmp/.s.PGSQL.54857"
2026-08-11 06:32:04.821 UTC [15896] LOG: database system was shut down at 2026-08-11 06:32:03 UTC
2026-08-11 06:32:04.826 UTC [15894] LOG: database system is ready to accept connections
I20260811 06:32:06.046669 29631 mini_postgres.cc:96] Postgres bound to 54857
2026-08-11 06:32:06.071 UTC [15908] FATAL: database "slave" does not exist
127.28.239.212:54857 - accepting connections
I20260811 06:32:06.071879 29631 mini_ranger.cc:161] Starting Ranger...
I20260811 06:32:06.106134 29631 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 06:32:06.623323 29631 mini_ranger.cc:87] Created ranger Postgres database
I20260811 06:32:06.623449 29631 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ranger-admin
2026-08-11 06:32:06,939 [I] DB FLAVOR :POSTGRES
2026-08-11 06:32:06,939 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 06:32:06,939 [I] Checking connection..
2026-08-11 06:32:06,939 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJveEHt/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJveEHt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.239.212:54857/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 06:32:07,237 [I] Checking connection passed.
2026-08-11 06:32:07,237 [I] --------- Verifying version history table ---------
2026-08-11 06:32:07,237 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJveEHt/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJveEHt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.239.212:54857/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 06:32:07,491 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 06:32:07,492 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJveEHt/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJveEHt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.239.212:54857/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 06:32:07,777 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 06:32:07,777 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 06:32:07,778 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJveEHt/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJveEHt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.239.212:54857/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskJveEHt/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 06:32:08.062 UTC [16013] WARNING: there is no transaction in progress
2026-08-11 06:32:08,072 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 06:32:08,073 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJveEHt/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJveEHt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.239.212:54857/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 06:32:08,354 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 06:32:08,354 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 06:32:08,354 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJveEHt/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJveEHt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.239.212:54857/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 06:32:08,617 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJveEHt/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJveEHt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.239.212:54857/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 06:32:08,874 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJveEHt/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJveEHt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.239.212:54857/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-gkw7.c.gcp-upstream.internal','N') ;"
2026-08-11 06:32:09,142 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJveEHt/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJveEHt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.239.212:54857/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 06:32:09,435 [I] Table x_portal_user does not exist in database ranger
2026-08-11 06:32:09,436 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJveEHt/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJveEHt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.239.212:54857/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 06:32:09,709 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 06:32:09,709 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJveEHt/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJveEHt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.239.212:54857/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 06:32:09,965 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJveEHt/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJveEHt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.239.212:54857/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 06:32:10,229 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 06:32:10,229 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJveEHt/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJveEHt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.239.212:54857/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskJveEHt/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 06:32:11.246 UTC [16190] WARNING: there is no transaction in progress
2026-08-11 06:32:11.263 UTC [16190] WARNING: there is no transaction in progress
2026-08-11 06:32:11.282 UTC [16190] WARNING: there is no transaction in progress
2026-08-11 06:32:11.297 UTC [16190] WARNING: there is no transaction in progress
2026-08-11 06:32:11.313 UTC [16190] WARNING: there is no transaction in progress
2026-08-11 06:32:11.329 UTC [16190] WARNING: there is no transaction in progress
2026-08-11 06:32:11.387 UTC [16190] WARNING: there is no transaction in progress
2026-08-11 06:32:11.394 UTC [16190] WARNING: there is no transaction in progress
2026-08-11 06:32:11.408 UTC [16190] WARNING: there is no transaction in progress
2026-08-11 06:32:11.419 UTC [16190] WARNING: there is no transaction in progress
2026-08-11 06:32:11.430 UTC [16190] WARNING: there is no transaction in progress
2026-08-11 06:32:11.444 UTC [16190] WARNING: there is no transaction in progress
2026-08-11 06:32:11.455 UTC [16190] WARNING: there is no transaction in progress
2026-08-11 06:32:11.462 UTC [16190] WARNING: there is no transaction in progress
2026-08-11 06:32:11.469 UTC [16190] WARNING: there is no transaction in progress
2026-08-11 06:32:11.476 UTC [16190] WARNING: there is no transaction in progress
2026-08-11 06:32:12,141 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 06:32:12,142 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJveEHt/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJveEHt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.239.212:54857/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 06:32:12,412 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 06:32:12,412 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJveEHt/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJveEHt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.239.212:54857/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 06:32:12,694 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 06:32:12,694 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJveEHt/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJveEHt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.239.212:54857/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 06:32:12,947 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 06:32:12,947 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJveEHt/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJveEHt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.239.212:54857/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 06:32:13,217 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJveEHt/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJveEHt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.239.212:54857/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 06:32:13,481 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJveEHt/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJveEHt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.239.212:54857/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-gkw7.c.gcp-upstream.internal';"
2026-08-11 06:32:13,727 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 06:32:13,728 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJveEHt/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJveEHt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.239.212:54857/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 06:32:14,007 [I] DB_PATCHES have already been applied
I20260811 06:32:14.014101 29631 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ranger-admin:/tmp/dist-test-taskJveEHt/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskJveEHt/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskJveEHt/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJveEHt/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 06:32:14.014219 29631 mini_ranger.cc:193] Using host: 127.28.239.212
I20260811 06:32:14.035604 29631 mini_ranger.cc:227] Ranger admin URL: http://127.28.239.212:33737
Aug 11, 2026 6:32:14 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 6:32:14 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 6:32:14 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 6:32:14 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 6:32:14 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 6:32:14 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 6:32:14 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 06:32:15.485806 29631 mini_ranger.cc:160] Time spent starting Ranger: real 9.414s user 0.000s sys 0.064s
Aug 11, 2026 6:32:35 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 6:32:35 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 11, 2026 6:32:35 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 6:32:35 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 6:32:35 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 6:32:35 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 6:32:35 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 6:32:35 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 6:32:35 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 6:32:35 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 6:32:35 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 6:32:35 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 6:32:35 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 6:32:35 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 6:32:35 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 6:32:35 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 6:32:35 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 6:32:36 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 06:32:37.403095 29631 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.28.239.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.28.239.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.28.239.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/krb5kdc/rangerkms_127.28.239.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.28.239.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/krb5kdc/rangerkms_127.28.239.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.28.239.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.28.239.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.28.239.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/krb5kdc/HTTP_127.28.239.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.28.239.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/krb5kdc/HTTP_127.28.239.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 06:32:37 dist-test-slave-gkw7 krb5kdc[15844](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429957, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 06:32:37.632802 29631 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 06:32:37.661010 29631 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 06:32:38.575054 29631 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 06:32:38.575204 29631 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ranger-kms
I20260811 06:32:38.575260 29631 mini_ranger_kms.cc:227] Using postgres at 127.28.239.212:54857
2026-08-11 06:32:38,963 [I] DB FLAVOR :POSTGRES
2026-08-11 06:32:38,963 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 06:32:38,964 [I] Checking connection
2026-08-11 06:32:38,964 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJveEHt/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.239.212:54857/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 06:32:39,221 [I] connection success
2026-08-11 06:32:39,221 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 06:32:39,221 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 06:32:39,222 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJveEHt/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.239.212:54857/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 06:32:39,466 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 06:32:39,466 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 06:32:39,466 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 06:32:39,466 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJveEHt/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.239.212:54857/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 06:32:39,745 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 06:32:40.264590 29631 mini_ranger_kms.cc:310] Created kms service
I20260811 06:32:40.481877 29631 mini_ranger_kms.cc:326] Created kudu user
I20260811 06:32:40.527972 29631 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 06:32:40 dist-test-slave-gkw7 krb5kdc[15844](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429957, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.28.239.212@KRBTEST.COM
I20260811 06:32:40.980336 29631 mini_ranger_kms.cc:384] Added ranger policy
I20260811 06:32:40.980700 29631 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ranger-kms:/tmp/dist-test-taskJveEHt/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJveEHt/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskJveEHt/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 06:32:40.980768 29631 mini_ranger_kms.cc:240] Using host: 127.28.239.212
I20260811 06:32:41.004256 29631 mini_ranger_kms.cc:276] Ranger KMS PID: 16461
I20260811 06:32:41.004405 29631 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.28.239.212:33339
Aug 11, 2026 6:32:42 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 6:32:42 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 6:32:42 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 6:32:42 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 6:32:42 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 6:32:42 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 11 06:32:43 dist-test-slave-gkw7 krb5kdc[15844](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786429963, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.28.239.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:32:43.867870 29631 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.235s user 0.086s sys 0.341s
I20260811 06:32:43.868130 29631 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 06:32:43.868317 29631 mini_ranger_kms.cc:401] 127.28.239.212:33339/kms/v1/keys
Aug 11 06:32:44 dist-test-slave-gkw7 krb5kdc[15844](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429963, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.28.239.212@KRBTEST.COM for HTTP/127.28.239.212@KRBTEST.COM
Aug 11 06:32:44 dist-test-slave-gkw7 krb5kdc[15844](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429963, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.28.239.212@KRBTEST.COM for HTTP/127.28.239.212@KRBTEST.COM
Aug 11 06:32:46 dist-test-slave-gkw7 krb5kdc[15844](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429966, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.28.239.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.239.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.239.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab.
Entry for principal kudu/127.28.239.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab.
I20260811 06:32:46.837833 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:42845
--webserver_interface=127.28.239.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab
--principal=kudu/127.28.239.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:34243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.239.254:42845
--ranger_config_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.28.239.212:33339/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:32:47.220511 16545 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:32:47.220842 16545 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:32:47.220991 16545 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:32:47.228713 16545 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:32:47.228839 16545 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:32:47.228884 16545 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:32:47.228997 16545 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:32:47.229089 16545 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:32:47.240099 16545 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:34243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/master-0/wal
--ranger_kms_url=127.28.239.212:33339/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.28.239.254:42845
--ranger_config_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab
--principal=kudu/127.28.239.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:42845
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.239.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:32:47.241707 16545 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:32:47.243223 16545 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:32:47.257627 16550 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:32:47.257671 16551 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:32:47.260944 16553 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:32:47.263334 16545 server_base.cc:1034] running on GCE node
I20260811 06:32:47.264386 16545 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:32:47.266417 16545 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:32:47.267659 16545 hybrid_clock.cc:648] HybridClock initialized: now 1786429967267595 us; error 59 us; skew 500 ppm
I20260811 06:32:47.268090 16545 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:32:47 dist-test-slave-gkw7 krb5kdc[15844](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429967, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:32:47.277115 16545 init.cc:377] Logged in from keytab as kudu/127.28.239.254@KRBTEST.COM (short username kudu)
I20260811 06:32:47.279639 16545 webserver.cc:460] Webserver started at http://127.28.239.254:38391/ using document root <none> and password file <none>
I20260811 06:32:47.280755 16545 fs_manager.cc:359] Metadata directory not provided
I20260811 06:32:47.280900 16545 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:32:47.281258 16545 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 06:32:47 dist-test-slave-gkw7 krb5kdc[15844](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429967, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.254@KRBTEST.COM for HTTP/127.28.239.212@KRBTEST.COM
I20260811 06:32:47.349301 16545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/master-0/data/instance:
uuid: "bb1821d155764d65854cdefc47f51bb6"
format_stamp: "Formatted at 2026-08-11 06:32:47 on dist-test-slave-gkw7"
server_key: "7c86ebaa9496f6865593d36e750a5650"
server_key_iv: "29ff47df5770e0ce15a26b62a4094539"
server_key_version: "kuduclusterkey@0"
I20260811 06:32:47.350293 16545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/master-0/wal/instance:
uuid: "bb1821d155764d65854cdefc47f51bb6"
format_stamp: "Formatted at 2026-08-11 06:32:47 on dist-test-slave-gkw7"
server_key: "7c86ebaa9496f6865593d36e750a5650"
server_key_iv: "29ff47df5770e0ce15a26b62a4094539"
server_key_version: "kuduclusterkey@0"
I20260811 06:32:47.356582 16545 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260811 06:32:47.377252 16565 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:32:47.379145 16545 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260811 06:32:47.379380 16545 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/master-0/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/master-0/wal
uuid: "bb1821d155764d65854cdefc47f51bb6"
format_stamp: "Formatted at 2026-08-11 06:32:47 on dist-test-slave-gkw7"
server_key: "7c86ebaa9496f6865593d36e750a5650"
server_key_iv: "29ff47df5770e0ce15a26b62a4094539"
server_key_version: "kuduclusterkey@0"
I20260811 06:32:47.379644 16545 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:32:47.399796 16545 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:32:47.404327 16545 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:32:47.404691 16545 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:32:47.426191 16545 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.254:42845
I20260811 06:32:47.426193 16641 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.254:42845 every 8 connection(s)
I20260811 06:32:47.427753 16545 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
I20260811 06:32:47.432415 16642 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:32:47.432758 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 16545
I20260811 06:32:47.433007 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/master-0/wal/instance
I20260811 06:32:47.433918 29631 external_mini_cluster.cc:1428] Setting key 56acc180bebcdcac7fb9f9445f207c7a
I20260811 06:32:47.456494 16642 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb1821d155764d65854cdefc47f51bb6: Bootstrap starting.
I20260811 06:32:47.460668 16642 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bb1821d155764d65854cdefc47f51bb6: Neither blocks nor log segments found. Creating new log.
I20260811 06:32:47.461922 16642 log.cc:824] T 00000000000000000000000000000000 P bb1821d155764d65854cdefc47f51bb6: Log is configured to *not* fsync() on all Append() calls
I20260811 06:32:47.465276 16642 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb1821d155764d65854cdefc47f51bb6: No bootstrap required, opened a new log
Aug 11 06:32:47 dist-test-slave-gkw7 krb5kdc[15844](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429966, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:32:47.476212 16642 raft_consensus.cc:348] T 00000000000000000000000000000000 P bb1821d155764d65854cdefc47f51bb6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb1821d155764d65854cdefc47f51bb6" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 42845 } }
I20260811 06:32:47.476718 16642 raft_consensus.cc:374] T 00000000000000000000000000000000 P bb1821d155764d65854cdefc47f51bb6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:32:47.476847 16642 raft_consensus.cc:729] T 00000000000000000000000000000000 P bb1821d155764d65854cdefc47f51bb6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb1821d155764d65854cdefc47f51bb6, State: Initialized, Role: FOLLOWER
I20260811 06:32:47.477664 16642 consensus_queue.cc:260] T 00000000000000000000000000000000 P bb1821d155764d65854cdefc47f51bb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb1821d155764d65854cdefc47f51bb6" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 42845 } }
I20260811 06:32:47.478055 16642 raft_consensus.cc:388] T 00000000000000000000000000000000 P bb1821d155764d65854cdefc47f51bb6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:32:47.478256 16642 raft_consensus.cc:482] T 00000000000000000000000000000000 P bb1821d155764d65854cdefc47f51bb6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:32:47.478479 16642 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bb1821d155764d65854cdefc47f51bb6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:32:47.480880 16645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:32:47.436279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60568 (local address 127.28.239.254:42845)
0811 06:32:47.454495 (+ 18216us) server_negotiation.cc:206] Beginning negotiation
0811 06:32:47.454513 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 06:32:47.454586 (+ 73us) server_negotiation.cc:407] Connection header received
0811 06:32:47.457091 (+ 2505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:32:47.457128 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:32:47.457748 (+ 620us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:32:47.458240 (+ 492us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:32:47.460058 (+ 1818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:32:47.462140 (+ 2082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:32:47.463862 (+ 1722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:32:47.464963 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:32:47.465154 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:32:47.470642 (+ 5488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:32:47.470656 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:32:47.470688 (+ 32us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:32:47.470748 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:32:47.475111 (+ 4363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:32:47.476060 (+ 949us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:32:47.476069 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:32:47.476078 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:32:47.476288 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:32:47.476788 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:32:47.476795 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:32:47.476799 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:32:47.477658 (+ 859us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:32:47.477994 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0811 06:32:47.478859 (+ 865us) server_negotiation.cc:299] Negotiation successful
0811 06:32:47.479336 (+ 477us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17917,"thread_start_us":17364,"threads_started":1}
I20260811 06:32:47.482214 16642 raft_consensus.cc:504] T 00000000000000000000000000000000 P bb1821d155764d65854cdefc47f51bb6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb1821d155764d65854cdefc47f51bb6" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 42845 } }
I20260811 06:32:47.482851 16642 leader_election.cc:302] T 00000000000000000000000000000000 P bb1821d155764d65854cdefc47f51bb6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bb1821d155764d65854cdefc47f51bb6; no voters:
I20260811 06:32:47.483637 16642 leader_election.cc:288] T 00000000000000000000000000000000 P bb1821d155764d65854cdefc47f51bb6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:32:47.484776 16648 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bb1821d155764d65854cdefc47f51bb6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:32:47.487710 16648 raft_consensus.cc:686] T 00000000000000000000000000000000 P bb1821d155764d65854cdefc47f51bb6 [term 1 LEADER]: Becoming Leader. State: Replica: bb1821d155764d65854cdefc47f51bb6, State: Running, Role: LEADER
I20260811 06:32:47.488528 16648 consensus_queue.cc:237] T 00000000000000000000000000000000 P bb1821d155764d65854cdefc47f51bb6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb1821d155764d65854cdefc47f51bb6" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 42845 } }
I20260811 06:32:47.490010 16642 sys_catalog.cc:564] T 00000000000000000000000000000000 P bb1821d155764d65854cdefc47f51bb6 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:32:47.493848 16651 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb1821d155764d65854cdefc47f51bb6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bb1821d155764d65854cdefc47f51bb6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb1821d155764d65854cdefc47f51bb6" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 42845 } } }
I20260811 06:32:47.494356 16651 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb1821d155764d65854cdefc47f51bb6 [sys.catalog]: This master's current role is: LEADER
I20260811 06:32:47.494974 16649 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb1821d155764d65854cdefc47f51bb6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bb1821d155764d65854cdefc47f51bb6. Latest consensus state: current_term: 1 leader_uuid: "bb1821d155764d65854cdefc47f51bb6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb1821d155764d65854cdefc47f51bb6" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 42845 } } }
I20260811 06:32:47.495316 16649 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb1821d155764d65854cdefc47f51bb6 [sys.catalog]: This master's current role is: LEADER
I20260811 06:32:47.495344 16655 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:32:47.505400 16655 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:32:47.525308 16655 catalog_manager.cc:1347] Generated new cluster ID: fecdefad25694fed8ed81716be12b983
I20260811 06:32:47.525478 16655 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:32:47.525279 16642 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 06:32:47.565886 16655 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:32:47.568377 16655 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:32:47.588562 16655 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bb1821d155764d65854cdefc47f51bb6: Generated new TSK 0
I20260811 06:32:47.590041 16655 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.239.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.239.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.239.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.193.keytab.
Entry for principal kudu/127.28.239.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.193.keytab.
I20260811 06:32:48.449060 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.28.239.212:33339/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.193:0
--local_ip_for_outbound_sockets=127.28.239.193
--webserver_interface=127.28.239.193
--webserver_port=0
--tserver_master_addrs=127.28.239.254:42845
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.193.keytab
--principal=kudu/127.28.239.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:34243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 06:32:48 dist-test-slave-gkw7 krb5kdc[15844](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786429968, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260811 06:32:48.890471 16704 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:32:48.890861 16704 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:32:48.891028 16704 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:32:48.898721 16704 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:32:48.898893 16704 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:32:48.899088 16704 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.193
I20260811 06:32:48.912168 16704 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:34243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.28.239.212:33339/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.193.keytab
--principal=kudu/127.28.239.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.239.193
--webserver_port=0
--tserver_master_addrs=127.28.239.254:42845
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.193
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:32:48.913865 16704 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:32:48.915269 16704 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:32:48.940138 16716 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:32:48.950448 16717 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:32:48.950640 16704 server_base.cc:1034] running on GCE node
W20260811 06:32:48.951748 16719 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:32:48.952607 16704 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:32:48.953824 16704 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:32:48.955096 16704 hybrid_clock.cc:648] HybridClock initialized: now 1786429968955024 us; error 68 us; skew 500 ppm
I20260811 06:32:48.955478 16704 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:32:48 dist-test-slave-gkw7 krb5kdc[15844](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429968, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:32:48.959122 16704 init.cc:377] Logged in from keytab as kudu/127.28.239.193@KRBTEST.COM (short username kudu)
I20260811 06:32:48.965920 16704 webserver.cc:460] Webserver started at http://127.28.239.193:36001/ using document root <none> and password file <none>
I20260811 06:32:48.966735 16704 fs_manager.cc:359] Metadata directory not provided
I20260811 06:32:48.966893 16704 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:32:48.967259 16704 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 06:32:48 dist-test-slave-gkw7 krb5kdc[15844](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429968, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.193@KRBTEST.COM for HTTP/127.28.239.212@KRBTEST.COM
I20260811 06:32:48.986253 16704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-0/data/instance:
uuid: "65886e78f7264109afdc44b79b4bf075"
format_stamp: "Formatted at 2026-08-11 06:32:48 on dist-test-slave-gkw7"
server_key: "c9e811b65c679f74b5a181208330eb71"
server_key_iv: "df3e19287590258c05edb98692da96f1"
server_key_version: "kuduclusterkey@0"
I20260811 06:32:48.987444 16704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-0/wal/instance:
uuid: "65886e78f7264109afdc44b79b4bf075"
format_stamp: "Formatted at 2026-08-11 06:32:48 on dist-test-slave-gkw7"
server_key: "c9e811b65c679f74b5a181208330eb71"
server_key_iv: "df3e19287590258c05edb98692da96f1"
server_key_version: "kuduclusterkey@0"
I20260811 06:32:49.006381 16704 fs_manager.cc:691] Time spent creating directory manager: real 0.018s user 0.005s sys 0.008s
I20260811 06:32:49.036756 16732 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:32:49.039073 16704 fs_manager.cc:725] Time spent opening block manager: real 0.013s user 0.006s sys 0.005s
I20260811 06:32:49.039314 16704 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-0/wal
uuid: "65886e78f7264109afdc44b79b4bf075"
format_stamp: "Formatted at 2026-08-11 06:32:48 on dist-test-slave-gkw7"
server_key: "c9e811b65c679f74b5a181208330eb71"
server_key_iv: "df3e19287590258c05edb98692da96f1"
server_key_version: "kuduclusterkey@0"
I20260811 06:32:49.039675 16704 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:32:49.067561 16704 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:32:49.073328 16704 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:32:49.073809 16704 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:32:49.091094 16704 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:32:49.099578 16704 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:32:49.099817 16704 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260811 06:32:49.100041 16704 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:32:49.100162 16704 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:32:49.228219 16704 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.193:37463
I20260811 06:32:49.228240 16888 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.193:37463 every 8 connection(s)
I20260811 06:32:49.229792 16704 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
I20260811 06:32:49.238302 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 16704
I20260811 06:32:49.238557 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-0/wal/instance
I20260811 06:32:49.239353 29631 external_mini_cluster.cc:1428] Setting key e3c23b9c764db55e9f8bab0aa91ac15b
Aug 11 06:32:49 dist-test-slave-gkw7 krb5kdc[15844](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429968, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.254@KRBTEST.COM for HTTP/127.28.239.212@KRBTEST.COM
Aug 11 06:32:49 dist-test-slave-gkw7 krb5kdc[15844](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429968, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.193@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:32:49.298427 16893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:32:49.236915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:42845 (local address 127.28.239.193:41871)
0811 06:32:49.239327 (+ 2412us) negotiation.cc:107] Waiting for socket to connect
0811 06:32:49.239388 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 06:32:49.241882 (+ 2494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:32:49.277823 (+ 35941us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:32:49.277840 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:32:49.278610 (+ 770us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:32:49.279613 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:32:49.279643 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:32:49.281314 (+ 1671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:32:49.281320 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:32:49.283511 (+ 2191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:32:49.283540 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:32:49.284933 (+ 1393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:32:49.284940 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:32:49.285243 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:32:49.286176 (+ 933us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:32:49.286208 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:32:49.289119 (+ 2911us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:32:49.293211 (+ 4092us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:32:49.293221 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:32:49.293242 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:32:49.293903 (+ 661us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:32:49.294396 (+ 493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:32:49.294401 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:32:49.294405 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:32:49.294651 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:32:49.295390 (+ 739us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:32:49.295399 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:32:49.295865 (+ 466us) client_negotiation.cc:769] Sending connection context
0811 06:32:49.296168 (+ 303us) client_negotiation.cc:240] Negotiation successful
0811 06:32:49.296813 (+ 645us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2083,"thread_start_us":3141,"threads_started":1}
I20260811 06:32:49.299620 16896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:32:49.243962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.193:41871 (local address 127.28.239.254:42845)
0811 06:32:49.273766 (+ 29804us) server_negotiation.cc:206] Beginning negotiation
0811 06:32:49.273775 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:32:49.273812 (+ 37us) server_negotiation.cc:407] Connection header received
0811 06:32:49.273900 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:32:49.273909 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:32:49.274031 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:32:49.274228 (+ 197us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:32:49.279888 (+ 5660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:32:49.281108 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:32:49.283763 (+ 2655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:32:49.284423 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:32:49.284583 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:32:49.289422 (+ 4839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:32:49.289429 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:32:49.289434 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:32:49.289479 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:32:49.293036 (+ 3557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:32:49.294077 (+ 1041us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:32:49.294082 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:32:49.294086 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:32:49.294257 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:32:49.294801 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:32:49.294807 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:32:49.294810 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:32:49.295165 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:32:49.295308 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 06:32:49.298995 (+ 3687us) server_negotiation.cc:299] Negotiation successful
0811 06:32:49.299254 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29604,"thread_start_us":30393,"threads_started":1}
I20260811 06:32:49.302170 16890 heartbeater.cc:347] Connected to a master server at 127.28.239.254:42845
I20260811 06:32:49.302580 16890 heartbeater.cc:464] Registering TS with master...
I20260811 06:32:49.303551 16890 heartbeater.cc:511] Master 127.28.239.254:42845 requested a full tablet report, sending...
I20260811 06:32:49.306334 16596 ts_manager.cc:194] Registered new tserver with Master: 65886e78f7264109afdc44b79b4bf075 (127.28.239.193:37463)
I20260811 06:32:49.308439 16596 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.239.193@KRBTEST.COM'} at 127.28.239.193:41871
WARNING: no policy specified for kudu/127.28.239.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.239.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.239.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.194.keytab.
Entry for principal kudu/127.28.239.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.194.keytab.
I20260811 06:32:49.370067 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.28.239.212:33339/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.194:0
--local_ip_for_outbound_sockets=127.28.239.194
--webserver_interface=127.28.239.194
--webserver_port=0
--tserver_master_addrs=127.28.239.254:42845
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.194.keytab
--principal=kudu/127.28.239.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:34243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 06:32:49 dist-test-slave-gkw7 krb5kdc[15844](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429968, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.254@KRBTEST.COM for HTTP/127.28.239.212@KRBTEST.COM
W20260811 06:32:49.846539 16900 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:32:49.847082 16900 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:32:49.847378 16900 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:32:49.858990 16900 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:32:49.859227 16900 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:32:49.859469 16900 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.194
I20260811 06:32:49.874836 16900 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:34243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.28.239.212:33339/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.194.keytab
--principal=kudu/127.28.239.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.239.194
--webserver_port=0
--tserver_master_addrs=127.28.239.254:42845
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.194
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:32:49.876900 16900 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:32:49.878432 16900 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:32:49.911952 16913 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:32:49.913913 16912 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:32:49.919430 16900 server_base.cc:1034] running on GCE node
W20260811 06:32:49.919844 16917 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:32:49.920846 16900 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:32:49.928474 16900 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:32:49.929827 16900 hybrid_clock.cc:648] HybridClock initialized: now 1786429969929731 us; error 102 us; skew 500 ppm
I20260811 06:32:49.930286 16900 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:32:49 dist-test-slave-gkw7 krb5kdc[15844](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429969, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:32:49.934748 16900 init.cc:377] Logged in from keytab as kudu/127.28.239.194@KRBTEST.COM (short username kudu)
I20260811 06:32:49.966722 16900 webserver.cc:460] Webserver started at http://127.28.239.194:46613/ using document root <none> and password file <none>
I20260811 06:32:49.967912 16900 fs_manager.cc:359] Metadata directory not provided
I20260811 06:32:49.968133 16900 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:32:49.968554 16900 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 06:32:49 dist-test-slave-gkw7 krb5kdc[15844](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429969, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.194@KRBTEST.COM for HTTP/127.28.239.212@KRBTEST.COM
I20260811 06:32:49.989413 16900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-1/data/instance:
uuid: "b3e3512ddef54b5399040e7caa50ebb5"
format_stamp: "Formatted at 2026-08-11 06:32:49 on dist-test-slave-gkw7"
server_key: "aeb6cf129eed7df4424afe0e77d54b1a"
server_key_iv: "fe0c77e5499c30fed0cd685faa3d939d"
server_key_version: "kuduclusterkey@0"
I20260811 06:32:49.990597 16900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-1/wal/instance:
uuid: "b3e3512ddef54b5399040e7caa50ebb5"
format_stamp: "Formatted at 2026-08-11 06:32:49 on dist-test-slave-gkw7"
server_key: "aeb6cf129eed7df4424afe0e77d54b1a"
server_key_iv: "fe0c77e5499c30fed0cd685faa3d939d"
server_key_version: "kuduclusterkey@0"
I20260811 06:32:50.013602 16900 fs_manager.cc:691] Time spent creating directory manager: real 0.021s user 0.012s sys 0.008s
I20260811 06:32:50.042335 16941 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:32:50.044552 16900 fs_manager.cc:725] Time spent opening block manager: real 0.013s user 0.005s sys 0.007s
I20260811 06:32:50.044826 16900 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-1/wal
uuid: "b3e3512ddef54b5399040e7caa50ebb5"
format_stamp: "Formatted at 2026-08-11 06:32:49 on dist-test-slave-gkw7"
server_key: "aeb6cf129eed7df4424afe0e77d54b1a"
server_key_iv: "fe0c77e5499c30fed0cd685faa3d939d"
server_key_version: "kuduclusterkey@0"
I20260811 06:32:50.045131 16900 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:32:50.073403 16900 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:32:50.079196 16900 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:32:50.079546 16900 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:32:50.080809 16900 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:32:50.082865 16900 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:32:50.083006 16900 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260811 06:32:50.083168 16900 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:32:50.083258 16900 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:32:50.129657 16900 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.194:36851
I20260811 06:32:50.129730 17110 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.194:36851 every 8 connection(s)
I20260811 06:32:50.131497 16900 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
I20260811 06:32:50.131992 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 16900
I20260811 06:32:50.132174 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-1/wal/instance
I20260811 06:32:50.132768 29631 external_mini_cluster.cc:1428] Setting key 849ce538b4c757de6860d4245dff6130
Aug 11 06:32:50 dist-test-slave-gkw7 krb5kdc[15844](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429969, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.194@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.28.239.195@KRBTEST.COM; defaulting to no policy
I20260811 06:32:50.176563 17119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:32:50.146496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.194:36137 (local address 127.28.239.254:42845)
0811 06:32:50.154006 (+ 7510us) server_negotiation.cc:206] Beginning negotiation
0811 06:32:50.154013 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:32:50.154884 (+ 871us) server_negotiation.cc:407] Connection header received
0811 06:32:50.157695 (+ 2811us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:32:50.157702 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:32:50.157809 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:32:50.157932 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:32:50.160176 (+ 2244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:32:50.161472 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:32:50.163843 (+ 2371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:32:50.164451 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:32:50.164548 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:32:50.169307 (+ 4759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:32:50.169313 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:32:50.169318 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:32:50.169358 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:32:50.172822 (+ 3464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:32:50.173773 (+ 951us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:32:50.173779 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:32:50.173782 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:32:50.173933 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:32:50.174440 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:32:50.174444 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:32:50.174447 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:32:50.174788 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:32:50.174925 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 06:32:50.175909 (+ 984us) server_negotiation.cc:299] Negotiation successful
0811 06:32:50.176148 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7359,"thread_start_us":7209,"threads_started":1}
I20260811 06:32:50.178759 17116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:32:50.145680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:42845 (local address 127.28.239.194:36137)
0811 06:32:50.154639 (+ 8959us) negotiation.cc:107] Waiting for socket to connect
0811 06:32:50.154716 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0811 06:32:50.157174 (+ 2458us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:32:50.158418 (+ 1244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:32:50.158431 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:32:50.159100 (+ 669us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:32:50.159963 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:32:50.159989 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:32:50.161690 (+ 1701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:32:50.161695 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:32:50.163667 (+ 1972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:32:50.163692 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:32:50.164912 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:32:50.164918 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:32:50.165205 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:32:50.166035 (+ 830us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:32:50.166064 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:32:50.169029 (+ 2965us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:32:50.172985 (+ 3956us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:32:50.172995 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:32:50.173014 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:32:50.173639 (+ 625us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:32:50.174074 (+ 435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:32:50.174078 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:32:50.174082 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:32:50.174317 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:32:50.175031 (+ 714us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:32:50.175039 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:32:50.175478 (+ 439us) client_negotiation.cc:769] Sending connection context
0811 06:32:50.176799 (+ 1321us) client_negotiation.cc:240] Negotiation successful
0811 06:32:50.177354 (+ 555us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8614,"thread_start_us":8353,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.239.195@KRBTEST.COM" created.
I20260811 06:32:50.180557 17112 heartbeater.cc:347] Connected to a master server at 127.28.239.254:42845
I20260811 06:32:50.180879 17112 heartbeater.cc:464] Registering TS with master...
I20260811 06:32:50.181761 17112 heartbeater.cc:511] Master 127.28.239.254:42845 requested a full tablet report, sending...
I20260811 06:32:50.183643 16596 ts_manager.cc:194] Registered new tserver with Master: b3e3512ddef54b5399040e7caa50ebb5 (127.28.239.194:36851)
I20260811 06:32:50.184772 16596 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.239.194@KRBTEST.COM'} at 127.28.239.194:36137
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.239.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.195.keytab.
Entry for principal kudu/127.28.239.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.195.keytab.
I20260811 06:32:50.220896 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.28.239.212:33339/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.195:0
--local_ip_for_outbound_sockets=127.28.239.195
--webserver_interface=127.28.239.195
--webserver_port=0
--tserver_master_addrs=127.28.239.254:42845
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.195.keytab
--principal=kudu/127.28.239.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:34243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 06:32:50.312374 16890 heartbeater.cc:503] Master 127.28.239.254:42845 was elected leader, sending a full tablet report...
W20260811 06:32:50.647857 17125 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:32:50.648208 17125 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:32:50.648394 17125 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:32:50.655999 17125 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:32:50.656121 17125 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:32:50.656339 17125 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.195
I20260811 06:32:50.666540 17125 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:34243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.28.239.212:33339/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.195.keytab
--principal=kudu/127.28.239.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.239.195
--webserver_port=0
--tserver_master_addrs=127.28.239.254:42845
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.195
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:32:50.667900 17125 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:32:50.669190 17125 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:32:50.683403 17131 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:32:50.683596 17138 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:32:50.683645 17141 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:32:50.685020 17125 server_base.cc:1034] running on GCE node
I20260811 06:32:50.685887 17125 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:32:50.687048 17125 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:32:50.688386 17125 hybrid_clock.cc:648] HybridClock initialized: now 1786429970688301 us; error 35 us; skew 500 ppm
I20260811 06:32:50.688966 17125 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:32:50 dist-test-slave-gkw7 krb5kdc[15844](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429970, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:32:50.694164 17125 init.cc:377] Logged in from keytab as kudu/127.28.239.195@KRBTEST.COM (short username kudu)
I20260811 06:32:50.696342 17125 webserver.cc:460] Webserver started at http://127.28.239.195:45533/ using document root <none> and password file <none>
I20260811 06:32:50.697613 17125 fs_manager.cc:359] Metadata directory not provided
I20260811 06:32:50.697808 17125 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:32:50.698303 17125 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 06:32:50 dist-test-slave-gkw7 krb5kdc[15844](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429970, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.195@KRBTEST.COM for HTTP/127.28.239.212@KRBTEST.COM
I20260811 06:32:50.719704 17125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-2/data/instance:
uuid: "73f130f9a7c4431387b722049ead6d39"
format_stamp: "Formatted at 2026-08-11 06:32:50 on dist-test-slave-gkw7"
server_key: "29f88273f39dba0d27ac62245d28d720"
server_key_iv: "db56ab16ae222e54d6e663f658316854"
server_key_version: "kuduclusterkey@0"
I20260811 06:32:50.720820 17125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-2/wal/instance:
uuid: "73f130f9a7c4431387b722049ead6d39"
format_stamp: "Formatted at 2026-08-11 06:32:50 on dist-test-slave-gkw7"
server_key: "29f88273f39dba0d27ac62245d28d720"
server_key_iv: "db56ab16ae222e54d6e663f658316854"
server_key_version: "kuduclusterkey@0"
I20260811 06:32:50.728929 17125 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.009s sys 0.000s
I20260811 06:32:50.753656 17149 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:32:50.755970 17125 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.001s
I20260811 06:32:50.756229 17125 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-2/wal
uuid: "73f130f9a7c4431387b722049ead6d39"
format_stamp: "Formatted at 2026-08-11 06:32:50 on dist-test-slave-gkw7"
server_key: "29f88273f39dba0d27ac62245d28d720"
server_key_iv: "db56ab16ae222e54d6e663f658316854"
server_key_version: "kuduclusterkey@0"
I20260811 06:32:50.756572 17125 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:32:50.774551 17125 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:32:50.778976 17125 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:32:50.779269 17125 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:32:50.780359 17125 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:32:50.781960 17125 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:32:50.782050 17125 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:32:50.782145 17125 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:32:50.782197 17125 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:32:50.817044 17125 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.195:36581
I20260811 06:32:50.817080 17324 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.195:36581 every 8 connection(s)
I20260811 06:32:50.818681 17125 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
I20260811 06:32:50.821218 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 17125
I20260811 06:32:50.821434 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-2/wal/instance
I20260811 06:32:50.822122 29631 external_mini_cluster.cc:1428] Setting key 03d2a859d9b790270d86480e7702fd0a
Aug 11 06:32:50 dist-test-slave-gkw7 krb5kdc[15844](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429970, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.195@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:32:50.840662 17329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:32:50.821590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.195:50881 (local address 127.28.239.254:42845)
0811 06:32:50.822251 (+ 661us) server_negotiation.cc:206] Beginning negotiation
0811 06:32:50.822260 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:32:50.823554 (+ 1294us) server_negotiation.cc:407] Connection header received
0811 06:32:50.825861 (+ 2307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:32:50.825866 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:32:50.825976 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:32:50.826103 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:32:50.827611 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:32:50.828401 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:32:50.830119 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:32:50.830580 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:32:50.830721 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:32:50.834296 (+ 3575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:32:50.834301 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:32:50.834305 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:32:50.834342 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:32:50.837243 (+ 2901us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:32:50.838167 (+ 924us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:32:50.838172 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:32:50.838174 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:32:50.838311 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:32:50.838862 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:32:50.838866 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:32:50.838868 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:32:50.839147 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:32:50.839312 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0811 06:32:50.840068 (+ 756us) server_negotiation.cc:299] Negotiation successful
0811 06:32:50.840297 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":462,"thread_start_us":310,"threads_started":1}
I20260811 06:32:50.841806 17331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:32:50.822383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:42845 (local address 127.28.239.195:50881)
0811 06:32:50.823356 (+ 973us) negotiation.cc:107] Waiting for socket to connect
0811 06:32:50.823416 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 06:32:50.825507 (+ 2091us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:32:50.826376 (+ 869us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:32:50.826384 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:32:50.826821 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:32:50.827436 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:32:50.827454 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:32:50.828563 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:32:50.828568 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:32:50.829967 (+ 1399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:32:50.829980 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:32:50.830835 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:32:50.830845 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:32:50.831115 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:32:50.831822 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:32:50.831846 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:32:50.834049 (+ 2203us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:32:50.837407 (+ 3358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:32:50.837417 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:32:50.837443 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:32:50.838025 (+ 582us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:32:50.838428 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:32:50.838433 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:32:50.838437 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:32:50.838722 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:32:50.839302 (+ 580us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:32:50.839309 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:32:50.839675 (+ 366us) client_negotiation.cc:769] Sending connection context
0811 06:32:50.839957 (+ 282us) client_negotiation.cc:240] Negotiation successful
0811 06:32:50.840533 (+ 576us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":641,"thread_start_us":412,"threads_started":1}
I20260811 06:32:50.843297 17326 heartbeater.cc:347] Connected to a master server at 127.28.239.254:42845
I20260811 06:32:50.843650 17326 heartbeater.cc:464] Registering TS with master...
I20260811 06:32:50.844404 17326 heartbeater.cc:511] Master 127.28.239.254:42845 requested a full tablet report, sending...
I20260811 06:32:50.846161 16596 ts_manager.cc:194] Registered new tserver with Master: 73f130f9a7c4431387b722049ead6d39 (127.28.239.195:36581)
I20260811 06:32:50.847167 16596 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.239.195@KRBTEST.COM'} at 127.28.239.195:50881
I20260811 06:32:50.851007 29631 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 06:32:50.852885 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 16704
Aug 11 06:32:50 dist-test-slave-gkw7 krb5kdc[15844](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429966, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.239.193@KRBTEST.COM
I20260811 06:32:50.870922 17333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:32:50.853958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56040 (local address 127.28.239.193:37463)
0811 06:32:50.854633 (+ 675us) server_negotiation.cc:206] Beginning negotiation
0811 06:32:50.854643 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:32:50.854677 (+ 34us) server_negotiation.cc:407] Connection header received
0811 06:32:50.854753 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:32:50.854760 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:32:50.854967 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:32:50.855159 (+ 192us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:32:50.856716 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:32:50.857818 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:32:50.859323 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:32:50.859896 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:32:50.860036 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:32:50.864838 (+ 4802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:32:50.864850 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:32:50.864866 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:32:50.864903 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:32:50.867848 (+ 2945us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:32:50.868619 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:32:50.868628 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:32:50.868630 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:32:50.868784 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:32:50.869150 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:32:50.869154 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:32:50.869156 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:32:50.869505 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:32:50.869918 (+ 413us) server_negotiation.cc:1035] Waiting for connection context
0811 06:32:50.870265 (+ 347us) server_negotiation.cc:299] Negotiation successful
0811 06:32:50.870476 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":470,"thread_start_us":291,"threads_started":1}
I20260811 06:32:50.872382 16884 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:32:50.957226 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 16704
I20260811 06:32:50.983289 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 16900
Aug 11 06:32:50 dist-test-slave-gkw7 krb5kdc[15844](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429966, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.239.194@KRBTEST.COM
I20260811 06:32:51.000828 17336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:32:50.984414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47904 (local address 127.28.239.194:36851)
0811 06:32:50.985024 (+ 610us) server_negotiation.cc:206] Beginning negotiation
0811 06:32:50.985031 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:32:50.985056 (+ 25us) server_negotiation.cc:407] Connection header received
0811 06:32:50.985118 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:32:50.985124 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:32:50.985354 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:32:50.985574 (+ 220us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:32:50.986934 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:32:50.987943 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:32:50.989391 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:32:50.990092 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:32:50.990259 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:32:50.994974 (+ 4715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:32:50.994986 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:32:50.995001 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:32:50.995036 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:32:50.997806 (+ 2770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:32:50.998563 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:32:50.998570 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:32:50.998573 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:32:50.998715 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:32:50.999101 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:32:50.999106 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:32:50.999110 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:32:50.999522 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:32:50.999828 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0811 06:32:51.000164 (+ 336us) server_negotiation.cc:299] Negotiation successful
0811 06:32:51.000453 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":467,"thread_start_us":338,"threads_started":1}
I20260811 06:32:51.002398 17105 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:32:51.088263 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 16900
I20260811 06:32:51.118839 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 17125
Aug 11 06:32:51 dist-test-slave-gkw7 krb5kdc[15844](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429966, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.239.195@KRBTEST.COM
I20260811 06:32:51.137681 17338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:32:51.120014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35662 (local address 127.28.239.195:36581)
0811 06:32:51.120758 (+ 744us) server_negotiation.cc:206] Beginning negotiation
0811 06:32:51.120766 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:32:51.120803 (+ 37us) server_negotiation.cc:407] Connection header received
0811 06:32:51.120872 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:32:51.120879 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:32:51.121136 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:32:51.121285 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:32:51.122869 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:32:51.124147 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:32:51.125679 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:32:51.126320 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:32:51.126443 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:32:51.131131 (+ 4688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:32:51.131146 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:32:51.131163 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:32:51.131211 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:32:51.134562 (+ 3351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:32:51.135470 (+ 908us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:32:51.135480 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:32:51.135482 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:32:51.135638 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:32:51.136000 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:32:51.136004 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:32:51.136006 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:32:51.136464 (+ 458us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:32:51.136724 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0811 06:32:51.137033 (+ 309us) server_negotiation.cc:299] Negotiation successful
0811 06:32:51.137288 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":582,"thread_start_us":388,"threads_started":1}
I20260811 06:32:51.139151 17319 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:32:51.224336 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 17125
I20260811 06:32:51.255980 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 16545
I20260811 06:32:51.257498 16635 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:32:51.369176 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 16545
I20260811 06:32:51.407811 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:42845
--webserver_interface=127.28.239.254
--webserver_port=38391
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab
--principal=kudu/127.28.239.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:34243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.239.254:42845
--ranger_config_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.28.239.212:33339/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.28.239.212:33339/kms with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:32:51.752913 17348 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:32:51.753281 17348 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:32:51.753450 17348 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:32:51.760756 17348 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:32:51.760918 17348 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:32:51.761036 17348 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:32:51.761163 17348 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:32:51.761248 17348 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:32:51.771888 17348 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:34243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.28.239.212:33339/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.28.239.254:42845
--ranger_config_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab
--principal=kudu/127.28.239.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:42845
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.239.254
--webserver_port=38391
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:32:51.773363 17348 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:32:51.774793 17348 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:32:51.787366 17361 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:32:51.787420 17357 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:32:51.787396 17358 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:32:51.788324 17348 server_base.cc:1034] running on GCE node
I20260811 06:32:51.789173 17348 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:32:51.790679 17348 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:32:51.791924 17348 hybrid_clock.cc:648] HybridClock initialized: now 1786429971791886 us; error 34 us; skew 500 ppm
I20260811 06:32:51.792330 17348 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:32:51 dist-test-slave-gkw7 krb5kdc[15844](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429971, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:32:51.795771 17348 init.cc:377] Logged in from keytab as kudu/127.28.239.254@KRBTEST.COM (short username kudu)
I20260811 06:32:51.797495 17348 webserver.cc:460] Webserver started at http://127.28.239.254:38391/ using document root <none> and password file <none>
I20260811 06:32:51.798254 17348 fs_manager.cc:359] Metadata directory not provided
I20260811 06:32:51.798370 17348 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 06:32:51 dist-test-slave-gkw7 krb5kdc[15844](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429971, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.254@KRBTEST.COM for HTTP/127.28.239.212@KRBTEST.COM
I20260811 06:32:51.894953 17348 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.003s sys 0.000s
I20260811 06:32:51.898495 17371 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:32:51.900172 17348 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 06:32:51.900408 17348 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/master-0/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/master-0/wal
uuid: "bb1821d155764d65854cdefc47f51bb6"
format_stamp: "Formatted at 2026-08-11 06:32:47 on dist-test-slave-gkw7"
server_key: "7c86ebaa9496f6865593d36e750a5650"
server_key_iv: "29ff47df5770e0ce15a26b62a4094539"
server_key_version: "kuduclusterkey@0"
I20260811 06:32:51.901458 17348 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:32:51.939194 17348 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:32:51.943684 17348 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:32:51.944021 17348 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:32:51.963644 17348 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.254:42845
I20260811 06:32:51.963665 17452 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.254:42845 every 8 connection(s)
I20260811 06:32:51.965384 17348 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
I20260811 06:32:51.968868 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 17348
I20260811 06:32:51.969810 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.28.239.212:33339/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.193:37463
--local_ip_for_outbound_sockets=127.28.239.193
--tserver_master_addrs=127.28.239.254:42845
--webserver_port=36001
--webserver_interface=127.28.239.193
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.193.keytab
--principal=kudu/127.28.239.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:34243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.28.239.212:33339/kms with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 06:32:51.972744 17453 sys_catalog.cc:263] Verifying existing consensus state
I20260811 06:32:51.976393 17453 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb1821d155764d65854cdefc47f51bb6: Bootstrap starting.
I20260811 06:32:51.996349 17453 log.cc:824] T 00000000000000000000000000000000 P bb1821d155764d65854cdefc47f51bb6: Log is configured to *not* fsync() on all Append() calls
I20260811 06:32:52.005748 17453 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb1821d155764d65854cdefc47f51bb6: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 06:32:52.006366 17453 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb1821d155764d65854cdefc47f51bb6: Bootstrap complete.
I20260811 06:32:52.015826 17453 raft_consensus.cc:348] T 00000000000000000000000000000000 P bb1821d155764d65854cdefc47f51bb6 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb1821d155764d65854cdefc47f51bb6" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 42845 } }
I20260811 06:32:52.016752 17453 raft_consensus.cc:729] T 00000000000000000000000000000000 P bb1821d155764d65854cdefc47f51bb6 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb1821d155764d65854cdefc47f51bb6, State: Initialized, Role: FOLLOWER
I20260811 06:32:52.017407 17453 consensus_queue.cc:260] T 00000000000000000000000000000000 P bb1821d155764d65854cdefc47f51bb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb1821d155764d65854cdefc47f51bb6" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 42845 } }
I20260811 06:32:52.017720 17453 raft_consensus.cc:388] T 00000000000000000000000000000000 P bb1821d155764d65854cdefc47f51bb6 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:32:52.017858 17453 raft_consensus.cc:482] T 00000000000000000000000000000000 P bb1821d155764d65854cdefc47f51bb6 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:32:52.018050 17453 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bb1821d155764d65854cdefc47f51bb6 [term 1 FOLLOWER]: Advancing to term 2
I20260811 06:32:52.021998 17453 raft_consensus.cc:504] T 00000000000000000000000000000000 P bb1821d155764d65854cdefc47f51bb6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb1821d155764d65854cdefc47f51bb6" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 42845 } }
I20260811 06:32:52.022518 17453 leader_election.cc:302] T 00000000000000000000000000000000 P bb1821d155764d65854cdefc47f51bb6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bb1821d155764d65854cdefc47f51bb6; no voters:
I20260811 06:32:52.023200 17453 leader_election.cc:288] T 00000000000000000000000000000000 P bb1821d155764d65854cdefc47f51bb6 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 06:32:52.023352 17457 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bb1821d155764d65854cdefc47f51bb6 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 06:32:52.024550 17457 raft_consensus.cc:686] T 00000000000000000000000000000000 P bb1821d155764d65854cdefc47f51bb6 [term 2 LEADER]: Becoming Leader. State: Replica: bb1821d155764d65854cdefc47f51bb6, State: Running, Role: LEADER
I20260811 06:32:52.025202 17457 consensus_queue.cc:237] T 00000000000000000000000000000000 P bb1821d155764d65854cdefc47f51bb6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb1821d155764d65854cdefc47f51bb6" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 42845 } }
I20260811 06:32:52.025559 17453 sys_catalog.cc:564] T 00000000000000000000000000000000 P bb1821d155764d65854cdefc47f51bb6 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:32:52.028132 17459 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb1821d155764d65854cdefc47f51bb6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bb1821d155764d65854cdefc47f51bb6. Latest consensus state: current_term: 2 leader_uuid: "bb1821d155764d65854cdefc47f51bb6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb1821d155764d65854cdefc47f51bb6" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 42845 } } }
I20260811 06:32:52.028621 17459 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb1821d155764d65854cdefc47f51bb6 [sys.catalog]: This master's current role is: LEADER
I20260811 06:32:52.029253 17458 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb1821d155764d65854cdefc47f51bb6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "bb1821d155764d65854cdefc47f51bb6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb1821d155764d65854cdefc47f51bb6" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 42845 } } }
I20260811 06:32:52.029639 17458 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb1821d155764d65854cdefc47f51bb6 [sys.catalog]: This master's current role is: LEADER
I20260811 06:32:52.034982 17465 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:32:52.042390 17465 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:32:52.044487 17465 catalog_manager.cc:1259] Loaded cluster ID: fecdefad25694fed8ed81716be12b983
I20260811 06:32:52.044725 17465 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:32:52.050160 17465 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:32:52.052178 17465 catalog_manager.cc:5879] T 00000000000000000000000000000000 P bb1821d155764d65854cdefc47f51bb6: Loaded TSK: 0
I20260811 06:32:52.054301 17465 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:32:52.059900 17453 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
W20260811 06:32:52.464829 17455 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:32:52.465399 17455 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:32:52.465654 17455 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:32:52.477468 17455 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:32:52.477715 17455 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:32:52.477965 17455 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.193
I20260811 06:32:52.494407 17455 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:34243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.28.239.212:33339/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.193.keytab
--principal=kudu/127.28.239.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.193:37463
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.239.193
--webserver_port=36001
--tserver_master_addrs=127.28.239.254:42845
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.193
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:32:52.496646 17455 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:32:52.498595 17455 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:32:52.522449 17508 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:32:52.524009 17507 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:32:52.524591 17511 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:32:52.525914 17455 server_base.cc:1034] running on GCE node
I20260811 06:32:52.526705 17455 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:32:52.527992 17455 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:32:52.529268 17455 hybrid_clock.cc:648] HybridClock initialized: now 1786429972529194 us; error 65 us; skew 500 ppm
I20260811 06:32:52.529682 17455 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:32:52 dist-test-slave-gkw7 krb5kdc[15844](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429972, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:32:52.538681 17455 init.cc:377] Logged in from keytab as kudu/127.28.239.193@KRBTEST.COM (short username kudu)
I20260811 06:32:52.540503 17455 webserver.cc:460] Webserver started at http://127.28.239.193:36001/ using document root <none> and password file <none>
I20260811 06:32:52.541290 17455 fs_manager.cc:359] Metadata directory not provided
I20260811 06:32:52.541462 17455 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 06:32:52 dist-test-slave-gkw7 krb5kdc[15844](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429972, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.193@KRBTEST.COM for HTTP/127.28.239.212@KRBTEST.COM
I20260811 06:32:52.570896 17455 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.003s sys 0.000s
I20260811 06:32:52.582279 17521 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:32:52.584498 17455 fs_manager.cc:725] Time spent opening block manager: real 0.012s user 0.005s sys 0.004s
I20260811 06:32:52.584797 17455 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-0/wal
uuid: "65886e78f7264109afdc44b79b4bf075"
format_stamp: "Formatted at 2026-08-11 06:32:48 on dist-test-slave-gkw7"
server_key: "c9e811b65c679f74b5a181208330eb71"
server_key_iv: "df3e19287590258c05edb98692da96f1"
server_key_version: "kuduclusterkey@0"
I20260811 06:32:52.586447 17455 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:32:52.615851 17455 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:32:52.620471 17455 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:32:52.620793 17455 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:32:52.621990 17455 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:32:52.623697 17455 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:32:52.623827 17455 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:32:52.623961 17455 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:32:52.624061 17455 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:32:52.689795 17455 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.193:37463
I20260811 06:32:52.689893 17693 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.193:37463 every 8 connection(s)
I20260811 06:32:52.691856 17455 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
I20260811 06:32:52.692785 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 17455
I20260811 06:32:52.693845 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.28.239.212:33339/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.194:36851
--local_ip_for_outbound_sockets=127.28.239.194
--tserver_master_addrs=127.28.239.254:42845
--webserver_port=46613
--webserver_interface=127.28.239.194
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.194.keytab
--principal=kudu/127.28.239.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:34243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.28.239.212:33339/kms with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 06:32:52 dist-test-slave-gkw7 krb5kdc[15844](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429972, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.193@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:32:52.741004 17697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:32:52.700662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:42845 (local address 127.28.239.193:39903)
0811 06:32:52.713787 (+ 13125us) negotiation.cc:107] Waiting for socket to connect
0811 06:32:52.713864 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0811 06:32:52.715958 (+ 2094us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:32:52.717740 (+ 1782us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:32:52.717752 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:32:52.718386 (+ 634us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:32:52.719009 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:32:52.719028 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:32:52.720908 (+ 1880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:32:52.720914 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:32:52.722408 (+ 1494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:32:52.722422 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:32:52.723597 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:32:52.723606 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:32:52.723902 (+ 296us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:32:52.724681 (+ 779us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:32:52.724713 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:32:52.727208 (+ 2495us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:32:52.733209 (+ 6001us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:32:52.733225 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:32:52.733245 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:32:52.734390 (+ 1145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:32:52.735001 (+ 611us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:32:52.735006 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:32:52.735010 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:32:52.735305 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:32:52.736956 (+ 1651us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:32:52.736965 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:32:52.737486 (+ 521us) client_negotiation.cc:769] Sending connection context
0811 06:32:52.737866 (+ 380us) client_negotiation.cc:240] Negotiation successful
0811 06:32:52.739540 (+ 1674us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12737,"thread_start_us":12512,"threads_started":1}
I20260811 06:32:52.740991 17696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:32:52.700576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.193:39903 (local address 127.28.239.254:42845)
0811 06:32:52.713230 (+ 12654us) server_negotiation.cc:206] Beginning negotiation
0811 06:32:52.713260 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0811 06:32:52.716314 (+ 3054us) server_negotiation.cc:407] Connection header received
0811 06:32:52.716705 (+ 391us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:32:52.716719 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:32:52.717112 (+ 393us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:32:52.717520 (+ 408us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:32:52.719273 (+ 1753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:32:52.720707 (+ 1434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:32:52.722585 (+ 1878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:32:52.723306 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:32:52.723539 (+ 233us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:32:52.727610 (+ 4071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:32:52.727623 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:32:52.727644 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:32:52.727715 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:32:52.732972 (+ 5257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:32:52.734605 (+ 1633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:32:52.734616 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:32:52.734625 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:32:52.734853 (+ 228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:32:52.735997 (+ 1144us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:32:52.736004 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:32:52.736008 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:32:52.736765 (+ 757us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:32:52.736979 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0811 06:32:52.738818 (+ 1839us) server_negotiation.cc:299] Negotiation successful
0811 06:32:52.739328 (+ 510us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12299,"thread_start_us":12048,"threads_started":1}
I20260811 06:32:52.745308 17694 heartbeater.cc:347] Connected to a master server at 127.28.239.254:42845
I20260811 06:32:52.745707 17694 heartbeater.cc:464] Registering TS with master...
I20260811 06:32:52.746730 17694 heartbeater.cc:511] Master 127.28.239.254:42845 requested a full tablet report, sending...
W20260811 06:32:52.750900 17694 heartbeater.cc:644] Failed to heartbeat to 127.28.239.254:42845 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 06:32:52.751190 17694 heartbeater.cc:464] Registering TS with master...
I20260811 06:32:52.751507 17694 heartbeater.cc:511] Master 127.28.239.254:42845 requested a full tablet report, sending...
I20260811 06:32:52.753090 17694 heartbeater.cc:464] Registering TS with master...
I20260811 06:32:52.753486 17694 heartbeater.cc:511] Master 127.28.239.254:42845 requested a full tablet report, sending...
W20260811 06:32:52.754891 17694 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 06:32:53.150221 17699 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:32:53.150622 17699 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:32:53.150825 17699 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:32:53.159865 17699 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:32:53.160027 17699 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:32:53.160223 17699 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.194
I20260811 06:32:53.171646 17699 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:34243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.28.239.212:33339/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.194.keytab
--principal=kudu/127.28.239.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.194:36851
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.239.194
--webserver_port=46613
--tserver_master_addrs=127.28.239.254:42845
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.194
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:32:53.173213 17699 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:32:53.175031 17699 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:32:53.192726 17720 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:32:53.193965 17721 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:32:53.199453 17699 server_base.cc:1034] running on GCE node
W20260811 06:32:53.199733 17724 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:32:53.200650 17699 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:32:53.202054 17699 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:32:53.203361 17699 hybrid_clock.cc:648] HybridClock initialized: now 1786429973203296 us; error 50 us; skew 500 ppm
I20260811 06:32:53.203828 17699 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:32:53 dist-test-slave-gkw7 krb5kdc[15844](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429973, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:32:53.207988 17699 init.cc:377] Logged in from keytab as kudu/127.28.239.194@KRBTEST.COM (short username kudu)
I20260811 06:32:53.210881 17699 webserver.cc:460] Webserver started at http://127.28.239.194:46613/ using document root <none> and password file <none>
I20260811 06:32:53.211643 17699 fs_manager.cc:359] Metadata directory not provided
I20260811 06:32:53.211812 17699 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 06:32:53 dist-test-slave-gkw7 krb5kdc[15844](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429973, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.194@KRBTEST.COM for HTTP/127.28.239.212@KRBTEST.COM
I20260811 06:32:53.234815 17699 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 06:32:53.238997 17734 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:32:53.243782 17699 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.003s sys 0.002s
I20260811 06:32:53.243990 17699 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-1/wal
uuid: "b3e3512ddef54b5399040e7caa50ebb5"
format_stamp: "Formatted at 2026-08-11 06:32:49 on dist-test-slave-gkw7"
server_key: "aeb6cf129eed7df4424afe0e77d54b1a"
server_key_iv: "fe0c77e5499c30fed0cd685faa3d939d"
server_key_version: "kuduclusterkey@0"
I20260811 06:32:53.245471 17699 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:32:53.286916 17699 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:32:53.292761 17699 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:32:53.293166 17699 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:32:53.294545 17699 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:32:53.306705 17699 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:32:53.306957 17699 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260811 06:32:53.307170 17699 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:32:53.307305 17699 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 06:32:53 dist-test-slave-gkw7 krb5kdc[15844](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786429973, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:32:53.755056 17694 heartbeater.cc:464] Registering TS with master...
I20260811 06:32:53.755537 17694 heartbeater.cc:511] Master 127.28.239.254:42845 requested a full tablet report, sending...
I20260811 06:32:53.757112 17397 ts_manager.cc:194] Registered new tserver with Master: 65886e78f7264109afdc44b79b4bf075 (127.28.239.193:37463)
I20260811 06:32:53.757874 17699 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.194:36851
I20260811 06:32:53.757929 17909 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.194:36851 every 8 connection(s)
I20260811 06:32:53.759414 17397 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.239.193@KRBTEST.COM'} at 127.28.239.193:39903
I20260811 06:32:53.759789 17699 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
I20260811 06:32:53.763029 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 17699
I20260811 06:32:53.764024 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.28.239.212:33339/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.195:36581
--local_ip_for_outbound_sockets=127.28.239.195
--tserver_master_addrs=127.28.239.254:42845
--webserver_port=45533
--webserver_interface=127.28.239.195
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.195.keytab
--principal=kudu/127.28.239.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:34243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.28.239.212:33339/kms with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 06:32:53 dist-test-slave-gkw7 krb5kdc[15844](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429973, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.194@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:32:53.794217 17914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:32:53.764146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.194:40159 (local address 127.28.239.254:42845)
0811 06:32:53.774033 (+ 9887us) server_negotiation.cc:206] Beginning negotiation
0811 06:32:53.774043 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:32:53.777067 (+ 3024us) server_negotiation.cc:407] Connection header received
0811 06:32:53.777131 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:32:53.777135 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:32:53.777205 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:32:53.777341 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:32:53.779663 (+ 2322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:32:53.780827 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:32:53.782611 (+ 1784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:32:53.783053 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:32:53.783203 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:32:53.787255 (+ 4052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:32:53.787261 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:32:53.787265 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:32:53.787303 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:32:53.790247 (+ 2944us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:32:53.791346 (+ 1099us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:32:53.791352 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:32:53.791355 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:32:53.791540 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:32:53.792052 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:32:53.792057 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:32:53.792061 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:32:53.792476 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:32:53.792695 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0811 06:32:53.793553 (+ 858us) server_negotiation.cc:299] Negotiation successful
0811 06:32:53.793796 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9695,"thread_start_us":9512,"threads_started":1}
I20260811 06:32:53.795380 17913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:32:53.763536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:42845 (local address 127.28.239.194:40159)
0811 06:32:53.774740 (+ 11204us) negotiation.cc:107] Waiting for socket to connect
0811 06:32:53.774802 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 06:32:53.776730 (+ 1928us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:32:53.777712 (+ 982us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:32:53.777731 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:32:53.778496 (+ 765us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:32:53.779388 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:32:53.779416 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:32:53.781020 (+ 1604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:32:53.781025 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:32:53.782444 (+ 1419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:32:53.782457 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:32:53.783333 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:32:53.783344 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:32:53.783604 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:32:53.784374 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:32:53.784404 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:32:53.786977 (+ 2573us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:32:53.790642 (+ 3665us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:32:53.790655 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:32:53.790676 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:32:53.791193 (+ 517us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:32:53.791674 (+ 481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:32:53.791679 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:32:53.791682 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:32:53.791917 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:32:53.792673 (+ 756us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:32:53.792683 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:32:53.793078 (+ 395us) client_negotiation.cc:769] Sending connection context
0811 06:32:53.793417 (+ 339us) client_negotiation.cc:240] Negotiation successful
0811 06:32:53.794034 (+ 617us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10868,"thread_start_us":10637,"threads_started":1}
I20260811 06:32:53.797094 17910 heartbeater.cc:347] Connected to a master server at 127.28.239.254:42845
I20260811 06:32:53.797454 17910 heartbeater.cc:464] Registering TS with master...
I20260811 06:32:53.798259 17910 heartbeater.cc:511] Master 127.28.239.254:42845 requested a full tablet report, sending...
I20260811 06:32:53.799794 17397 ts_manager.cc:194] Registered new tserver with Master: b3e3512ddef54b5399040e7caa50ebb5 (127.28.239.194:36851)
I20260811 06:32:53.800693 17397 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.239.194@KRBTEST.COM'} at 127.28.239.194:40159
Aug 11 06:32:53 dist-test-slave-gkw7 krb5kdc[15844](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429973, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.254@KRBTEST.COM for HTTP/127.28.239.212@KRBTEST.COM
W20260811 06:32:54.304324 17915 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:32:54.304816 17915 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:32:54.305074 17915 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:32:54.317041 17915 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:32:54.317219 17915 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:32:54.317464 17915 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.195
Aug 11 06:32:54 dist-test-slave-gkw7 krb5kdc[15844](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429973, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.254@KRBTEST.COM for HTTP/127.28.239.212@KRBTEST.COM
I20260811 06:32:54.335592 17915 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:34243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.28.239.212:33339/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.195.keytab
--principal=kudu/127.28.239.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.195:36581
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.239.195
--webserver_port=45533
--tserver_master_addrs=127.28.239.254:42845
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.195
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:32:54.337895 17915 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:32:54.340004 17915 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:32:54.403702 17927 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:32:54.409662 17928 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:32:54.424476 17915 server_base.cc:1034] running on GCE node
W20260811 06:32:54.425725 17930 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:32:54.426945 17915 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:32:54.442054 17915 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:32:54.443475 17915 hybrid_clock.cc:648] HybridClock initialized: now 1786429974443398 us; error 117 us; skew 500 ppm
I20260811 06:32:54.444100 17915 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:32:54 dist-test-slave-gkw7 krb5kdc[15844](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429974, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:32:54.449635 17915 init.cc:377] Logged in from keytab as kudu/127.28.239.195@KRBTEST.COM (short username kudu)
I20260811 06:32:54.494149 17915 webserver.cc:460] Webserver started at http://127.28.239.195:45533/ using document root <none> and password file <none>
I20260811 06:32:54.495724 17915 fs_manager.cc:359] Metadata directory not provided
I20260811 06:32:54.496035 17915 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 06:32:54 dist-test-slave-gkw7 krb5kdc[15844](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429974, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.195@KRBTEST.COM for HTTP/127.28.239.212@KRBTEST.COM
I20260811 06:32:54.558563 17915 fs_manager.cc:709] Time spent opening directory manager: real 0.018s user 0.011s sys 0.003s
I20260811 06:32:54.573098 17942 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:32:54.575672 17915 fs_manager.cc:725] Time spent opening block manager: real 0.014s user 0.007s sys 0.000s
I20260811 06:32:54.576042 17915 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-2/wal
uuid: "73f130f9a7c4431387b722049ead6d39"
format_stamp: "Formatted at 2026-08-11 06:32:50 on dist-test-slave-gkw7"
server_key: "29f88273f39dba0d27ac62245d28d720"
server_key_iv: "db56ab16ae222e54d6e663f658316854"
server_key_version: "kuduclusterkey@0"
I20260811 06:32:54.577896 17915 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:32:54.636256 17915 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:32:54.642768 17915 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:32:54.643298 17915 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:32:54.651160 17915 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:32:54.659456 17915 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:32:54.659680 17915 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:32:54.659843 17915 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:32:54.659956 17915 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:32:54.713601 17915 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.195:36581
I20260811 06:32:54.713685 18121 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.195:36581 every 8 connection(s)
I20260811 06:32:54.715517 17915 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
I20260811 06:32:54.719672 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 17915
Aug 11 06:32:54 dist-test-slave-gkw7 krb5kdc[15844](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429974, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.195@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:32:54.753075 18124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:32:54.720910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60578 (local address 127.28.239.254:42845)
0811 06:32:54.730638 (+ 9728us) server_negotiation.cc:206] Beginning negotiation
0811 06:32:54.730644 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:32:54.734403 (+ 3759us) server_negotiation.cc:407] Connection header received
0811 06:32:54.734476 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:32:54.734481 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:32:54.734604 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:32:54.734711 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:32:54.736885 (+ 2174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:32:54.737991 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:32:54.739989 (+ 1998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:32:54.740628 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:32:54.740743 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:32:54.744764 (+ 4021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:32:54.744771 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:32:54.744775 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:32:54.744819 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:32:54.748764 (+ 3945us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:32:54.749908 (+ 1144us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:32:54.749913 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:32:54.749917 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:32:54.750093 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:32:54.750593 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:32:54.750599 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:32:54.750603 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:32:54.750958 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:32:54.751239 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0811 06:32:54.752457 (+ 1218us) server_negotiation.cc:299] Negotiation successful
0811 06:32:54.752701 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9598,"thread_start_us":7516,"threads_started":1}
I20260811 06:32:54.754202 18131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:32:54.718651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.195:50547 (local address 127.28.239.254:42845)
0811 06:32:54.728693 (+ 10042us) server_negotiation.cc:206] Beginning negotiation
0811 06:32:54.728700 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:32:54.733611 (+ 4911us) server_negotiation.cc:407] Connection header received
0811 06:32:54.735780 (+ 2169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:32:54.735786 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:32:54.735870 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:32:54.735978 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:32:54.738302 (+ 2324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:32:54.739513 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:32:54.741956 (+ 2443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:32:54.742601 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:32:54.742687 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:32:54.747089 (+ 4402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:32:54.747095 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:32:54.747099 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:32:54.747167 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:32:54.750501 (+ 3334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:32:54.751394 (+ 893us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:32:54.751398 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:32:54.751401 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:32:54.751537 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:32:54.752108 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:32:54.752113 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:32:54.752116 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:32:54.752430 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:32:54.752548 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 06:32:54.753625 (+ 1077us) server_negotiation.cc:299] Negotiation successful
0811 06:32:54.753834 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9892,"thread_start_us":11684,"threads_started":1}
I20260811 06:32:54.756476 18126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:32:54.719214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:42845 (local address 127.28.239.195:50547)
0811 06:32:54.731310 (+ 12096us) negotiation.cc:107] Waiting for socket to connect
0811 06:32:54.731376 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0811 06:32:54.735326 (+ 3950us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:32:54.736499 (+ 1173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:32:54.736512 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:32:54.737140 (+ 628us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:32:54.738029 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:32:54.738057 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:32:54.739732 (+ 1675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:32:54.739737 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:32:54.741737 (+ 2000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:32:54.741764 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:32:54.743012 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:32:54.743018 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:32:54.743381 (+ 363us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:32:54.744100 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:32:54.744134 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:32:54.746809 (+ 2675us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:32:54.750667 (+ 3858us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:32:54.750676 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:32:54.750693 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:32:54.751263 (+ 570us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:32:54.751684 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:32:54.751688 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:32:54.751691 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:32:54.751976 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:32:54.752640 (+ 664us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:32:54.752648 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:32:54.753111 (+ 463us) client_negotiation.cc:769] Sending connection context
0811 06:32:54.754478 (+ 1367us) client_negotiation.cc:240] Negotiation successful
0811 06:32:54.755058 (+ 580us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11709,"thread_start_us":14721,"threads_started":1}
I20260811 06:32:54.758221 18122 heartbeater.cc:347] Connected to a master server at 127.28.239.254:42845
I20260811 06:32:54.758554 18122 heartbeater.cc:464] Registering TS with master...
I20260811 06:32:54.759450 18122 heartbeater.cc:511] Master 127.28.239.254:42845 requested a full tablet report, sending...
I20260811 06:32:54.761076 17397 ts_manager.cc:194] Registered new tserver with Master: 73f130f9a7c4431387b722049ead6d39 (127.28.239.195:36581)
I20260811 06:32:54.762212 17397 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.239.195@KRBTEST.COM'} at 127.28.239.195:50547
I20260811 06:32:54.764937 17694 heartbeater.cc:503] Master 127.28.239.254:42845 was elected leader, sending a full tablet report...
I20260811 06:32:54.768680 29631 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 06:32:54.787424 18124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:32:54.772409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60582 (local address 127.28.239.254:42845)
0811 06:32:54.772615 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 06:32:54.772621 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:32:54.773421 (+ 800us) server_negotiation.cc:407] Connection header received
0811 06:32:54.773654 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:32:54.773667 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:32:54.773743 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:32:54.773852 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:32:54.775441 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:32:54.776157 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:32:54.777610 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:32:54.778037 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:32:54.778172 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:32:54.780966 (+ 2794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:32:54.780975 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:32:54.780980 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:32:54.781026 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:32:54.784320 (+ 3294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:32:54.785146 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:32:54.785153 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:32:54.785157 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:32:54.785355 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:32:54.785871 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:32:54.785876 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:32:54.785880 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:32:54.786174 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:32:54.786458 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0811 06:32:54.786824 (+ 366us) server_negotiation.cc:299] Negotiation successful
0811 06:32:54.787063 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 06:32:54.793358 17397 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60582:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:32:54.801096 17397 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:32:54.804164 17910 heartbeater.cc:503] Master 127.28.239.254:42845 was elected leader, sending a full tablet report...
I20260811 06:32:54.859035 18152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:32:54.817722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47912 (local address 127.28.239.194:36851)
0811 06:32:54.849512 (+ 31790us) server_negotiation.cc:206] Beginning negotiation
0811 06:32:54.849519 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:32:54.849598 (+ 79us) server_negotiation.cc:407] Connection header received
0811 06:32:54.849732 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:32:54.849742 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:32:54.850057 (+ 315us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:32:54.850328 (+ 271us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:32:54.851470 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:32:54.853211 (+ 1741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:32:54.856111 (+ 2900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:32:54.857661 (+ 1550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:32:54.857852 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:32:54.858062 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0811 06:32:54.858518 (+ 456us) server_negotiation.cc:299] Negotiation successful
0811 06:32:54.858691 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31636,"thread_start_us":33583,"threads_started":1}
I20260811 06:32:54.859045 18153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:32:54.816899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.194:36851 (local address 127.0.0.1:47912)
0811 06:32:54.843545 (+ 26646us) negotiation.cc:107] Waiting for socket to connect
0811 06:32:54.843606 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 06:32:54.843906 (+ 300us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:32:54.850671 (+ 6765us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:32:54.850682 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:32:54.850722 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:32:54.851206 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:32:54.851220 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:32:54.853388 (+ 2168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:32:54.853395 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:32:54.855892 (+ 2497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:32:54.855906 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:32:54.858101 (+ 2195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:32:54.858108 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:32:54.858268 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:32:54.858304 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 06:32:54.858459 (+ 155us) client_negotiation.cc:240] Negotiation successful
0811 06:32:54.858674 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26305,"thread_start_us":34196,"threads_started":1}
I20260811 06:32:54.862814 18150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:32:54.817803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.193:37463 (local address 127.0.0.1:56050)
0811 06:32:54.850646 (+ 32843us) negotiation.cc:107] Waiting for socket to connect
0811 06:32:54.850664 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 06:32:54.851141 (+ 477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:32:54.853049 (+ 1908us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:32:54.853053 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:32:54.853077 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:32:54.853514 (+ 437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:32:54.853557 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:32:54.855565 (+ 2008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:32:54.855570 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:32:54.859619 (+ 4049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:32:54.859633 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:32:54.862006 (+ 2373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:32:54.862011 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:32:54.862178 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:32:54.862208 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 06:32:54.862284 (+ 76us) client_negotiation.cc:240] Negotiation successful
0811 06:32:54.862412 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32719,"mutex_wait_us":1637,"thread_start_us":26356,"threads_started":1}
I20260811 06:32:54.863543 18155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:32:54.818476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56050 (local address 127.28.239.193:37463)
0811 06:32:54.842585 (+ 24109us) server_negotiation.cc:206] Beginning negotiation
0811 06:32:54.842597 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 06:32:54.850834 (+ 8237us) server_negotiation.cc:407] Connection header received
0811 06:32:54.852329 (+ 1495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:32:54.852337 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:32:54.852627 (+ 290us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:32:54.852825 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:32:54.853802 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:32:54.855392 (+ 1590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:32:54.859827 (+ 4435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:32:54.861354 (+ 1527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:32:54.861558 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:32:54.861759 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0811 06:32:54.863061 (+ 1302us) server_negotiation.cc:299] Negotiation successful
0811 06:32:54.863239 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23886,"thread_start_us":24133,"threads_started":1}
I20260811 06:32:54.864190 18154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:32:54.818069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35666 (local address 127.28.239.195:36581)
0811 06:32:54.841160 (+ 23091us) server_negotiation.cc:206] Beginning negotiation
0811 06:32:54.841173 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 06:32:54.850173 (+ 9000us) server_negotiation.cc:407] Connection header received
0811 06:32:54.851391 (+ 1218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:32:54.851401 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:32:54.851772 (+ 371us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:32:54.851976 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:32:54.852729 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:32:54.854743 (+ 2014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:32:54.859484 (+ 4741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:32:54.860919 (+ 1435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:32:54.861127 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:32:54.861331 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0811 06:32:54.863676 (+ 2345us) server_negotiation.cc:299] Negotiation successful
0811 06:32:54.863853 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22862,"thread_start_us":22686,"threads_started":1}
I20260811 06:32:54.864632 18149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:32:54.817277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.195:36581 (local address 127.0.0.1:35666)
0811 06:32:54.850014 (+ 32737us) negotiation.cc:107] Waiting for socket to connect
0811 06:32:54.850035 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 06:32:54.850952 (+ 917us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:32:54.852087 (+ 1135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:32:54.852093 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:32:54.852124 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:32:54.852432 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:32:54.852443 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:32:54.855896 (+ 3453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:32:54.855901 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:32:54.858153 (+ 2252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:32:54.858168 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:32:54.861599 (+ 3431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:32:54.861606 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:32:54.861839 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:32:54.861883 (+ 44us) client_negotiation.cc:769] Sending connection context
0811 06:32:54.862001 (+ 118us) client_negotiation.cc:240] Negotiation successful
0811 06:32:54.862183 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32606,"mutex_wait_us":5,"thread_start_us":33082,"threads_started":1}
I20260811 06:32:54.863998 17813 tablet_service.cc:1467] Processing CreateTablet for tablet 8b87b5ce9678404d9f05062890bd0f9e (DEFAULT_TABLE table=test-table [id=a7208acf5db242069686226ba6919109]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:32:54.866609 17813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b87b5ce9678404d9f05062890bd0f9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:32:54.867789 17600 tablet_service.cc:1467] Processing CreateTablet for tablet 8b87b5ce9678404d9f05062890bd0f9e (DEFAULT_TABLE table=test-table [id=a7208acf5db242069686226ba6919109]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:32:54.868228 18023 tablet_service.cc:1467] Processing CreateTablet for tablet 8b87b5ce9678404d9f05062890bd0f9e (DEFAULT_TABLE table=test-table [id=a7208acf5db242069686226ba6919109]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:32:54.869928 17600 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b87b5ce9678404d9f05062890bd0f9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:32:54.870225 18023 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b87b5ce9678404d9f05062890bd0f9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:32:54.889657 18157 tablet_bootstrap.cc:492] T 8b87b5ce9678404d9f05062890bd0f9e P b3e3512ddef54b5399040e7caa50ebb5: Bootstrap starting.
I20260811 06:32:54.890723 18159 tablet_bootstrap.cc:492] T 8b87b5ce9678404d9f05062890bd0f9e P 73f130f9a7c4431387b722049ead6d39: Bootstrap starting.
I20260811 06:32:54.895587 18157 tablet_bootstrap.cc:654] T 8b87b5ce9678404d9f05062890bd0f9e P b3e3512ddef54b5399040e7caa50ebb5: Neither blocks nor log segments found. Creating new log.
I20260811 06:32:54.897069 18157 log.cc:824] T 8b87b5ce9678404d9f05062890bd0f9e P b3e3512ddef54b5399040e7caa50ebb5: Log is configured to *not* fsync() on all Append() calls
I20260811 06:32:54.897315 18159 tablet_bootstrap.cc:654] T 8b87b5ce9678404d9f05062890bd0f9e P 73f130f9a7c4431387b722049ead6d39: Neither blocks nor log segments found. Creating new log.
I20260811 06:32:54.898986 18159 log.cc:824] T 8b87b5ce9678404d9f05062890bd0f9e P 73f130f9a7c4431387b722049ead6d39: Log is configured to *not* fsync() on all Append() calls
I20260811 06:32:54.902354 18158 tablet_bootstrap.cc:492] T 8b87b5ce9678404d9f05062890bd0f9e P 65886e78f7264109afdc44b79b4bf075: Bootstrap starting.
I20260811 06:32:54.906188 18158 tablet_bootstrap.cc:654] T 8b87b5ce9678404d9f05062890bd0f9e P 65886e78f7264109afdc44b79b4bf075: Neither blocks nor log segments found. Creating new log.
I20260811 06:32:54.907567 18158 log.cc:824] T 8b87b5ce9678404d9f05062890bd0f9e P 65886e78f7264109afdc44b79b4bf075: Log is configured to *not* fsync() on all Append() calls
I20260811 06:32:54.908466 18157 tablet_bootstrap.cc:492] T 8b87b5ce9678404d9f05062890bd0f9e P b3e3512ddef54b5399040e7caa50ebb5: No bootstrap required, opened a new log
I20260811 06:32:54.908609 18159 tablet_bootstrap.cc:492] T 8b87b5ce9678404d9f05062890bd0f9e P 73f130f9a7c4431387b722049ead6d39: No bootstrap required, opened a new log
I20260811 06:32:54.908799 18157 ts_tablet_manager.cc:1397] T 8b87b5ce9678404d9f05062890bd0f9e P b3e3512ddef54b5399040e7caa50ebb5: Time spent bootstrapping tablet: real 0.019s user 0.013s sys 0.003s
I20260811 06:32:54.908975 18159 ts_tablet_manager.cc:1397] T 8b87b5ce9678404d9f05062890bd0f9e P 73f130f9a7c4431387b722049ead6d39: Time spent bootstrapping tablet: real 0.018s user 0.003s sys 0.010s
I20260811 06:32:54.917229 18158 tablet_bootstrap.cc:492] T 8b87b5ce9678404d9f05062890bd0f9e P 65886e78f7264109afdc44b79b4bf075: No bootstrap required, opened a new log
I20260811 06:32:54.917672 18158 ts_tablet_manager.cc:1397] T 8b87b5ce9678404d9f05062890bd0f9e P 65886e78f7264109afdc44b79b4bf075: Time spent bootstrapping tablet: real 0.016s user 0.014s sys 0.000s
I20260811 06:32:54.920842 18159 raft_consensus.cc:348] T 8b87b5ce9678404d9f05062890bd0f9e P 73f130f9a7c4431387b722049ead6d39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3e3512ddef54b5399040e7caa50ebb5" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 36851 } } peers { permanent_uuid: "73f130f9a7c4431387b722049ead6d39" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 36581 } } peers { permanent_uuid: "65886e78f7264109afdc44b79b4bf075" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 37463 } }
I20260811 06:32:54.920627 18157 raft_consensus.cc:348] T 8b87b5ce9678404d9f05062890bd0f9e P b3e3512ddef54b5399040e7caa50ebb5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3e3512ddef54b5399040e7caa50ebb5" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 36851 } } peers { permanent_uuid: "73f130f9a7c4431387b722049ead6d39" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 36581 } } peers { permanent_uuid: "65886e78f7264109afdc44b79b4bf075" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 37463 } }
I20260811 06:32:54.921376 18159 raft_consensus.cc:374] T 8b87b5ce9678404d9f05062890bd0f9e P 73f130f9a7c4431387b722049ead6d39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:32:54.921501 18157 raft_consensus.cc:374] T 8b87b5ce9678404d9f05062890bd0f9e P b3e3512ddef54b5399040e7caa50ebb5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:32:54.921517 18159 raft_consensus.cc:729] T 8b87b5ce9678404d9f05062890bd0f9e P 73f130f9a7c4431387b722049ead6d39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73f130f9a7c4431387b722049ead6d39, State: Initialized, Role: FOLLOWER
I20260811 06:32:54.921656 18157 raft_consensus.cc:729] T 8b87b5ce9678404d9f05062890bd0f9e P b3e3512ddef54b5399040e7caa50ebb5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3e3512ddef54b5399040e7caa50ebb5, State: Initialized, Role: FOLLOWER
I20260811 06:32:54.922379 18157 consensus_queue.cc:260] T 8b87b5ce9678404d9f05062890bd0f9e P b3e3512ddef54b5399040e7caa50ebb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3e3512ddef54b5399040e7caa50ebb5" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 36851 } } peers { permanent_uuid: "73f130f9a7c4431387b722049ead6d39" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 36581 } } peers { permanent_uuid: "65886e78f7264109afdc44b79b4bf075" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 37463 } }
I20260811 06:32:54.922374 18159 consensus_queue.cc:260] T 8b87b5ce9678404d9f05062890bd0f9e P 73f130f9a7c4431387b722049ead6d39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3e3512ddef54b5399040e7caa50ebb5" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 36851 } } peers { permanent_uuid: "73f130f9a7c4431387b722049ead6d39" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 36581 } } peers { permanent_uuid: "65886e78f7264109afdc44b79b4bf075" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 37463 } }
I20260811 06:32:54.931722 18158 raft_consensus.cc:348] T 8b87b5ce9678404d9f05062890bd0f9e P 65886e78f7264109afdc44b79b4bf075 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3e3512ddef54b5399040e7caa50ebb5" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 36851 } } peers { permanent_uuid: "73f130f9a7c4431387b722049ead6d39" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 36581 } } peers { permanent_uuid: "65886e78f7264109afdc44b79b4bf075" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 37463 } }
I20260811 06:32:54.934422 18157 ts_tablet_manager.cc:1428] T 8b87b5ce9678404d9f05062890bd0f9e P b3e3512ddef54b5399040e7caa50ebb5: Time spent starting tablet: real 0.025s user 0.011s sys 0.010s
I20260811 06:32:54.934553 18159 ts_tablet_manager.cc:1428] T 8b87b5ce9678404d9f05062890bd0f9e P 73f130f9a7c4431387b722049ead6d39: Time spent starting tablet: real 0.025s user 0.016s sys 0.008s
I20260811 06:32:54.934837 18158 raft_consensus.cc:374] T 8b87b5ce9678404d9f05062890bd0f9e P 65886e78f7264109afdc44b79b4bf075 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:32:54.935017 18158 raft_consensus.cc:729] T 8b87b5ce9678404d9f05062890bd0f9e P 65886e78f7264109afdc44b79b4bf075 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65886e78f7264109afdc44b79b4bf075, State: Initialized, Role: FOLLOWER
I20260811 06:32:54.935134 18122 heartbeater.cc:503] Master 127.28.239.254:42845 was elected leader, sending a full tablet report...
I20260811 06:32:54.935918 18158 consensus_queue.cc:260] T 8b87b5ce9678404d9f05062890bd0f9e P 65886e78f7264109afdc44b79b4bf075 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3e3512ddef54b5399040e7caa50ebb5" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 36851 } } peers { permanent_uuid: "73f130f9a7c4431387b722049ead6d39" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 36581 } } peers { permanent_uuid: "65886e78f7264109afdc44b79b4bf075" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 37463 } }
I20260811 06:32:54.941687 18158 ts_tablet_manager.cc:1428] T 8b87b5ce9678404d9f05062890bd0f9e P 65886e78f7264109afdc44b79b4bf075: Time spent starting tablet: real 0.024s user 0.013s sys 0.008s
W20260811 06:32:54.951786 17695 tablet.cc:2369] T 8b87b5ce9678404d9f05062890bd0f9e P 65886e78f7264109afdc44b79b4bf075: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 06:32:54.968798 18123 tablet.cc:2369] T 8b87b5ce9678404d9f05062890bd0f9e P 73f130f9a7c4431387b722049ead6d39: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 06:32:55.029443 17911 tablet.cc:2369] T 8b87b5ce9678404d9f05062890bd0f9e P b3e3512ddef54b5399040e7caa50ebb5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:32:55.106807 18170 raft_consensus.cc:482] T 8b87b5ce9678404d9f05062890bd0f9e P b3e3512ddef54b5399040e7caa50ebb5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:32:55.107262 18170 raft_consensus.cc:504] T 8b87b5ce9678404d9f05062890bd0f9e P b3e3512ddef54b5399040e7caa50ebb5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3e3512ddef54b5399040e7caa50ebb5" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 36851 } } peers { permanent_uuid: "73f130f9a7c4431387b722049ead6d39" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 36581 } } peers { permanent_uuid: "65886e78f7264109afdc44b79b4bf075" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 37463 } }
I20260811 06:32:55.109282 18170 leader_election.cc:288] T 8b87b5ce9678404d9f05062890bd0f9e P b3e3512ddef54b5399040e7caa50ebb5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 73f130f9a7c4431387b722049ead6d39 (127.28.239.195:36581), 65886e78f7264109afdc44b79b4bf075 (127.28.239.193:37463)
I20260811 06:32:55.118063 18155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:32:55.109869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.194:50461 (local address 127.28.239.193:37463)
0811 06:32:55.110146 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0811 06:32:55.110156 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:32:55.111273 (+ 1117us) server_negotiation.cc:407] Connection header received
0811 06:32:55.111532 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:32:55.111537 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:32:55.111639 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:32:55.111755 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:32:55.112530 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:32:55.113717 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:32:55.115789 (+ 2072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:32:55.117027 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:32:55.117214 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:32:55.117299 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 06:32:55.117615 (+ 316us) server_negotiation.cc:299] Negotiation successful
0811 06:32:55.117765 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 06:32:55.118165 18174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:32:55.109844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.193:37463 (local address 127.28.239.194:50461)
0811 06:32:55.111169 (+ 1325us) negotiation.cc:107] Waiting for socket to connect
0811 06:32:55.111190 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 06:32:55.111373 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:32:55.111785 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:32:55.111790 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:32:55.111825 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:32:55.112320 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:32:55.112336 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:32:55.113875 (+ 1539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:32:55.113892 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:32:55.115594 (+ 1702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:32:55.115610 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:32:55.117267 (+ 1657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:32:55.117272 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:32:55.117396 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:32:55.117423 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 06:32:55.117515 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 06:32:55.117706 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1197,"mutex_wait_us":3,"thread_start_us":508,"threads_started":1}
I20260811 06:32:55.118846 18154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:32:55.110160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.194:56163 (local address 127.28.239.195:36581)
0811 06:32:55.110366 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 06:32:55.110373 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:32:55.110945 (+ 572us) server_negotiation.cc:407] Connection header received
0811 06:32:55.111325 (+ 380us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:32:55.111333 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:32:55.111423 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:32:55.111526 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:32:55.112226 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:32:55.113459 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:32:55.116350 (+ 2891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:32:55.117740 (+ 1390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:32:55.117915 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:32:55.117987 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 06:32:55.118459 (+ 472us) server_negotiation.cc:299] Negotiation successful
0811 06:32:55.118604 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 06:32:55.118916 18175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:32:55.109507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.195:36581 (local address 127.28.239.194:56163)
0811 06:32:55.110808 (+ 1301us) negotiation.cc:107] Waiting for socket to connect
0811 06:32:55.110837 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0811 06:32:55.111114 (+ 277us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:32:55.111559 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:32:55.111564 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:32:55.111589 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:32:55.112055 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:32:55.112067 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:32:55.113642 (+ 1575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:32:55.113649 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:32:55.116142 (+ 2493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:32:55.116160 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:32:55.117981 (+ 1821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:32:55.117987 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:32:55.118213 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:32:55.118246 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 06:32:55.118426 (+ 180us) client_negotiation.cc:240] Negotiation successful
0811 06:32:55.118558 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1107,"mutex_wait_us":56,"thread_start_us":1228,"threads_started":1}
I20260811 06:32:55.119400 17628 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b87b5ce9678404d9f05062890bd0f9e" candidate_uuid: "b3e3512ddef54b5399040e7caa50ebb5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65886e78f7264109afdc44b79b4bf075" is_pre_election: true
I20260811 06:32:55.119967 17628 raft_consensus.cc:2454] T 8b87b5ce9678404d9f05062890bd0f9e P 65886e78f7264109afdc44b79b4bf075 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3e3512ddef54b5399040e7caa50ebb5 in term 0.
I20260811 06:32:55.120002 18054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b87b5ce9678404d9f05062890bd0f9e" candidate_uuid: "b3e3512ddef54b5399040e7caa50ebb5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73f130f9a7c4431387b722049ead6d39" is_pre_election: true
I20260811 06:32:55.120482 18054 raft_consensus.cc:2454] T 8b87b5ce9678404d9f05062890bd0f9e P 73f130f9a7c4431387b722049ead6d39 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3e3512ddef54b5399040e7caa50ebb5 in term 0.
I20260811 06:32:55.120795 17737 leader_election.cc:302] T 8b87b5ce9678404d9f05062890bd0f9e P b3e3512ddef54b5399040e7caa50ebb5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 65886e78f7264109afdc44b79b4bf075, b3e3512ddef54b5399040e7caa50ebb5; no voters:
I20260811 06:32:55.121296 18170 raft_consensus.cc:2781] T 8b87b5ce9678404d9f05062890bd0f9e P b3e3512ddef54b5399040e7caa50ebb5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:32:55.121465 18170 raft_consensus.cc:482] T 8b87b5ce9678404d9f05062890bd0f9e P b3e3512ddef54b5399040e7caa50ebb5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:32:55.121613 18170 raft_consensus.cc:3037] T 8b87b5ce9678404d9f05062890bd0f9e P b3e3512ddef54b5399040e7caa50ebb5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:32:55.123855 18170 raft_consensus.cc:504] T 8b87b5ce9678404d9f05062890bd0f9e P b3e3512ddef54b5399040e7caa50ebb5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3e3512ddef54b5399040e7caa50ebb5" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 36851 } } peers { permanent_uuid: "73f130f9a7c4431387b722049ead6d39" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 36581 } } peers { permanent_uuid: "65886e78f7264109afdc44b79b4bf075" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 37463 } }
I20260811 06:32:55.124390 18170 leader_election.cc:288] T 8b87b5ce9678404d9f05062890bd0f9e P b3e3512ddef54b5399040e7caa50ebb5 [CANDIDATE]: Term 1 election: Requested vote from peers 73f130f9a7c4431387b722049ead6d39 (127.28.239.195:36581), 65886e78f7264109afdc44b79b4bf075 (127.28.239.193:37463)
I20260811 06:32:55.124960 18054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b87b5ce9678404d9f05062890bd0f9e" candidate_uuid: "b3e3512ddef54b5399040e7caa50ebb5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73f130f9a7c4431387b722049ead6d39"
I20260811 06:32:55.124971 17628 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b87b5ce9678404d9f05062890bd0f9e" candidate_uuid: "b3e3512ddef54b5399040e7caa50ebb5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65886e78f7264109afdc44b79b4bf075"
I20260811 06:32:55.125183 17628 raft_consensus.cc:3037] T 8b87b5ce9678404d9f05062890bd0f9e P 65886e78f7264109afdc44b79b4bf075 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:32:55.125211 18054 raft_consensus.cc:3037] T 8b87b5ce9678404d9f05062890bd0f9e P 73f130f9a7c4431387b722049ead6d39 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:32:55.127661 17628 raft_consensus.cc:2454] T 8b87b5ce9678404d9f05062890bd0f9e P 65886e78f7264109afdc44b79b4bf075 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3e3512ddef54b5399040e7caa50ebb5 in term 1.
I20260811 06:32:55.127671 18054 raft_consensus.cc:2454] T 8b87b5ce9678404d9f05062890bd0f9e P 73f130f9a7c4431387b722049ead6d39 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3e3512ddef54b5399040e7caa50ebb5 in term 1.
I20260811 06:32:55.128093 17737 leader_election.cc:302] T 8b87b5ce9678404d9f05062890bd0f9e P b3e3512ddef54b5399040e7caa50ebb5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 65886e78f7264109afdc44b79b4bf075, b3e3512ddef54b5399040e7caa50ebb5; no voters:
I20260811 06:32:55.128391 18170 raft_consensus.cc:2781] T 8b87b5ce9678404d9f05062890bd0f9e P b3e3512ddef54b5399040e7caa50ebb5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:32:55.128863 18170 raft_consensus.cc:686] T 8b87b5ce9678404d9f05062890bd0f9e P b3e3512ddef54b5399040e7caa50ebb5 [term 1 LEADER]: Becoming Leader. State: Replica: b3e3512ddef54b5399040e7caa50ebb5, State: Running, Role: LEADER
I20260811 06:32:55.129446 18170 consensus_queue.cc:237] T 8b87b5ce9678404d9f05062890bd0f9e P b3e3512ddef54b5399040e7caa50ebb5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3e3512ddef54b5399040e7caa50ebb5" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 36851 } } peers { permanent_uuid: "73f130f9a7c4431387b722049ead6d39" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 36581 } } peers { permanent_uuid: "65886e78f7264109afdc44b79b4bf075" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 37463 } }
I20260811 06:32:55.134516 17396 catalog_manager.cc:5495] T 8b87b5ce9678404d9f05062890bd0f9e P b3e3512ddef54b5399040e7caa50ebb5 reported cstate change: term changed from 0 to 1, leader changed from <none> to b3e3512ddef54b5399040e7caa50ebb5 (127.28.239.194). New cstate: current_term: 1 leader_uuid: "b3e3512ddef54b5399040e7caa50ebb5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3e3512ddef54b5399040e7caa50ebb5" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 36851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "73f130f9a7c4431387b722049ead6d39" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 36581 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "65886e78f7264109afdc44b79b4bf075" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 37463 } health_report { overall_health: UNKNOWN } } }
I20260811 06:32:55.174716 18152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:32:55.168819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47928 (local address 127.28.239.194:36851)
0811 06:32:55.169085 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0811 06:32:55.169092 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:32:55.169238 (+ 146us) server_negotiation.cc:407] Connection header received
0811 06:32:55.169500 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:32:55.169504 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:32:55.169636 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:32:55.169799 (+ 163us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:32:55.170191 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:32:55.170928 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:32:55.172410 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:32:55.172823 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:32:55.172961 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:32:55.173489 (+ 528us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:32:55.174092 (+ 603us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:32:55.174286 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0811 06:32:55.174355 (+ 69us) server_negotiation.cc:299] Negotiation successful
0811 06:32:55.174438 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 06:32:55.183094 17628 raft_consensus.cc:1261] T 8b87b5ce9678404d9f05062890bd0f9e P 65886e78f7264109afdc44b79b4bf075 [term 1 FOLLOWER]: Refusing update from remote peer b3e3512ddef54b5399040e7caa50ebb5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:32:55.183161 18054 raft_consensus.cc:1261] T 8b87b5ce9678404d9f05062890bd0f9e P 73f130f9a7c4431387b722049ead6d39 [term 1 FOLLOWER]: Refusing update from remote peer b3e3512ddef54b5399040e7caa50ebb5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:32:55.184125 18177 consensus_queue.cc:1035] T 8b87b5ce9678404d9f05062890bd0f9e P b3e3512ddef54b5399040e7caa50ebb5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "65886e78f7264109afdc44b79b4bf075" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 37463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:32:55.184504 18170 consensus_queue.cc:1035] T 8b87b5ce9678404d9f05062890bd0f9e P b3e3512ddef54b5399040e7caa50ebb5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73f130f9a7c4431387b722049ead6d39" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 36581 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:32:55.196354 18181 mvcc.cc:205] Tried to move back new op lower bound from 7317217178339852288 to 7317217178134761472. Current Snapshot: MvccSnapshot[applied={T|T < 7317217178339852288}]
I20260811 06:32:55.198277 18186 mvcc.cc:205] Tried to move back new op lower bound from 7317217178339852288 to 7317217178134761472. Current Snapshot: MvccSnapshot[applied={T|T < 7317217178339852288}]
I20260811 06:32:55.212422 17396 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60582:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:32:55.212766 17396 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60582:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:32:55.216467 17396 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bb1821d155764d65854cdefc47f51bb6: Sending DeleteTablet for 3 replicas of tablet 8b87b5ce9678404d9f05062890bd0f9e
I20260811 06:32:55.217628 17813 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b87b5ce9678404d9f05062890bd0f9e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:32:55 UTC) from {username='kudu', principal='kudu/127.28.239.254@KRBTEST.COM'} at 127.0.0.1:47912
I20260811 06:32:55.217815 17600 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b87b5ce9678404d9f05062890bd0f9e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:32:55 UTC) from {username='kudu', principal='kudu/127.28.239.254@KRBTEST.COM'} at 127.0.0.1:56050
I20260811 06:32:55.218075 18023 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b87b5ce9678404d9f05062890bd0f9e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:32:55 UTC) from {username='kudu', principal='kudu/127.28.239.254@KRBTEST.COM'} at 127.0.0.1:35666
I20260811 06:32:55.219712 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 17455
I20260811 06:32:55.230365 18198 tablet_replica.cc:330] T 8b87b5ce9678404d9f05062890bd0f9e P 65886e78f7264109afdc44b79b4bf075: stopping tablet replica
I20260811 06:32:55.231024 18198 raft_consensus.cc:2229] T 8b87b5ce9678404d9f05062890bd0f9e P 65886e78f7264109afdc44b79b4bf075 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:32:55.231688 18198 raft_consensus.cc:2258] T 8b87b5ce9678404d9f05062890bd0f9e P 65886e78f7264109afdc44b79b4bf075 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:32:55.243577 18199 tablet_replica.cc:330] T 8b87b5ce9678404d9f05062890bd0f9e P 73f130f9a7c4431387b722049ead6d39: stopping tablet replica
I20260811 06:32:55.243896 18196 tablet_replica.cc:330] T 8b87b5ce9678404d9f05062890bd0f9e P b3e3512ddef54b5399040e7caa50ebb5: stopping tablet replica
I20260811 06:32:55.244273 18199 raft_consensus.cc:2229] T 8b87b5ce9678404d9f05062890bd0f9e P 73f130f9a7c4431387b722049ead6d39 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:32:55.244477 18196 raft_consensus.cc:2229] T 8b87b5ce9678404d9f05062890bd0f9e P b3e3512ddef54b5399040e7caa50ebb5 [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:32:55.244910 18199 raft_consensus.cc:2258] T 8b87b5ce9678404d9f05062890bd0f9e P 73f130f9a7c4431387b722049ead6d39 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:32:55.245363 18196 raft_consensus.cc:2258] T 8b87b5ce9678404d9f05062890bd0f9e P b3e3512ddef54b5399040e7caa50ebb5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:32:55.246237 18198 ts_tablet_manager.cc:1905] T 8b87b5ce9678404d9f05062890bd0f9e P 65886e78f7264109afdc44b79b4bf075: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:32:55.247663 18196 ts_tablet_manager.cc:1905] T 8b87b5ce9678404d9f05062890bd0f9e P b3e3512ddef54b5399040e7caa50ebb5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:32:55.250582 18155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:32:55.221103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56058 (local address 127.28.239.193:37463)
0811 06:32:55.221261 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 06:32:55.221267 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:32:55.221292 (+ 25us) server_negotiation.cc:407] Connection header received
0811 06:32:55.221368 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:32:55.221376 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:32:55.221460 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:32:55.221594 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:32:55.228309 (+ 6715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:32:55.229981 (+ 1672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:32:55.233297 (+ 3316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:32:55.234231 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:32:55.234395 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:32:55.239337 (+ 4942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:32:55.239361 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:32:55.239390 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:32:55.239452 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:32:55.245953 (+ 6501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:32:55.246953 (+ 1000us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:32:55.246965 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:32:55.246971 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:32:55.247153 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:32:55.247673 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:32:55.247680 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:32:55.247685 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:32:55.248233 (+ 548us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:32:55.248586 (+ 353us) server_negotiation.cc:1035] Waiting for connection context
0811 06:32:55.249877 (+ 1291us) server_negotiation.cc:299] Negotiation successful
0811 06:32:55.250157 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 06:32:55.251472 17686 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:32:55.252381 18199 ts_tablet_manager.cc:1905] T 8b87b5ce9678404d9f05062890bd0f9e P 73f130f9a7c4431387b722049ead6d39: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:32:55.255515 18196 ts_tablet_manager.cc:1918] T 8b87b5ce9678404d9f05062890bd0f9e P b3e3512ddef54b5399040e7caa50ebb5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:32:55.255790 18196 log.cc:1196] T 8b87b5ce9678404d9f05062890bd0f9e P b3e3512ddef54b5399040e7caa50ebb5: Deleting WAL directory at /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-1/wal/wals/8b87b5ce9678404d9f05062890bd0f9e
I20260811 06:32:55.256481 18196 ts_tablet_manager.cc:1939] T 8b87b5ce9678404d9f05062890bd0f9e P b3e3512ddef54b5399040e7caa50ebb5: Deleting consensus metadata
I20260811 06:32:55.258641 17379 catalog_manager.cc:4841] TS b3e3512ddef54b5399040e7caa50ebb5 (127.28.239.194:36851): tablet 8b87b5ce9678404d9f05062890bd0f9e (table test-table [id=a7208acf5db242069686226ba6919109]) successfully deleted
I20260811 06:32:55.260635 18199 ts_tablet_manager.cc:1918] T 8b87b5ce9678404d9f05062890bd0f9e P 73f130f9a7c4431387b722049ead6d39: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:32:55.260825 18199 log.cc:1196] T 8b87b5ce9678404d9f05062890bd0f9e P 73f130f9a7c4431387b722049ead6d39: Deleting WAL directory at /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-2/wal/wals/8b87b5ce9678404d9f05062890bd0f9e
I20260811 06:32:55.261382 18199 ts_tablet_manager.cc:1939] T 8b87b5ce9678404d9f05062890bd0f9e P 73f130f9a7c4431387b722049ead6d39: Deleting consensus metadata
I20260811 06:32:55.263011 17380 catalog_manager.cc:4841] TS 73f130f9a7c4431387b722049ead6d39 (127.28.239.195:36581): tablet 8b87b5ce9678404d9f05062890bd0f9e (table test-table [id=a7208acf5db242069686226ba6919109]) successfully deleted
I20260811 06:32:55.381763 18198 ts_tablet_manager.cc:1918] T 8b87b5ce9678404d9f05062890bd0f9e P 65886e78f7264109afdc44b79b4bf075: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:32:55.382058 18198 log.cc:1196] T 8b87b5ce9678404d9f05062890bd0f9e P 65886e78f7264109afdc44b79b4bf075: Deleting WAL directory at /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786429755927508-29631-0/minicluster-data/ts-0/wal/wals/8b87b5ce9678404d9f05062890bd0f9e
I20260811 06:32:55.382575 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 17455
I20260811 06:32:55.382758 18198 ts_tablet_manager.cc:1939] T 8b87b5ce9678404d9f05062890bd0f9e P 65886e78f7264109afdc44b79b4bf075: Deleting consensus metadata
I20260811 06:32:55.415355 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 17699
W20260811 06:32:55.415349 17378 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.28.239.193:37463 (error 108)
I20260811 06:32:55.417897 18205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:32:55.416954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.193:37463 (local address 127.0.0.1:56060)
0811 06:32:55.417612 (+ 658us) negotiation.cc:107] Waiting for socket to connect
0811 06:32:55.417715 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.193:37463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":407,"threads_started":1}
W20260811 06:32:55.418198 17378 catalog_manager.cc:4584] TS 65886e78f7264109afdc44b79b4bf075 (127.28.239.193:37463): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8b87b5ce9678404d9f05062890bd0f9e: Network error: Client connection negotiation failed: client connection to 127.28.239.193:37463: connect: Connection refused (error 111)
I20260811 06:32:55.432925 18152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:32:55.416527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47930 (local address 127.28.239.194:36851)
0811 06:32:55.416762 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0811 06:32:55.416768 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:32:55.416787 (+ 19us) server_negotiation.cc:407] Connection header received
0811 06:32:55.416857 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:32:55.416862 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:32:55.416942 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:32:55.417059 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:32:55.418931 (+ 1872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:32:55.419787 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:32:55.421806 (+ 2019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:32:55.422281 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:32:55.422391 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:32:55.425380 (+ 2989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:32:55.425396 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:32:55.425422 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:32:55.425472 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:32:55.429724 (+ 4252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:32:55.430536 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:32:55.430548 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:32:55.430552 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:32:55.430745 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:32:55.431177 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:32:55.431184 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:32:55.431187 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:32:55.431671 (+ 484us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:32:55.432020 (+ 349us) server_negotiation.cc:1035] Waiting for connection context
0811 06:32:55.432251 (+ 231us) server_negotiation.cc:299] Negotiation successful
0811 06:32:55.432517 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260811 06:32:55.433864 17904 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:32:55.471982 18205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:32:55.471382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.193:37463 (local address 127.0.0.1:56074)
0811 06:32:55.471639 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0811 06:32:55.471793 (+ 154us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.193:37463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260811 06:32:55.473245 18205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:32:55.472817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.193:37463 (local address 127.0.0.1:56084)
0811 06:32:55.472994 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0811 06:32:55.473084 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.193:37463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260811 06:32:55.534403 18205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:32:55.533870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.193:37463 (local address 127.0.0.1:56086)
0811 06:32:55.534103 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0811 06:32:55.534215 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.193:37463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260811 06:32:55.535596 18205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:32:55.535169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.193:37463 (local address 127.0.0.1:56098)
0811 06:32:55.535349 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0811 06:32:55.535432 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.193:37463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260811 06:32:55.551196 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 17699
I20260811 06:32:55.582314 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 17915
I20260811 06:32:55.597893 18154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:32:55.583231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35680 (local address 127.28.239.195:36581)
0811 06:32:55.583451 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0811 06:32:55.583457 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:32:55.583478 (+ 21us) server_negotiation.cc:407] Connection header received
0811 06:32:55.583738 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:32:55.583747 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:32:55.583836 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:32:55.583931 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:32:55.585360 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:32:55.586251 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:32:55.587681 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:32:55.588064 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:32:55.588173 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:32:55.591697 (+ 3524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:32:55.591722 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:32:55.591740 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:32:55.591794 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:32:55.595017 (+ 3223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:32:55.595728 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:32:55.595736 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:32:55.595738 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:32:55.595877 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:32:55.596281 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:32:55.596284 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:32:55.596287 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:32:55.596657 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:32:55.596898 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0811 06:32:55.597260 (+ 362us) server_negotiation.cc:299] Negotiation successful
0811 06:32:55.597448 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260811 06:32:55.598536 18115 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:32:55.616189 18205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:32:55.615705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.193:37463 (local address 127.0.0.1:56112)
0811 06:32:55.615921 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0811 06:32:55.616010 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.193:37463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260811 06:32:55.617359 18205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:32:55.616937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.193:37463 (local address 127.0.0.1:56116)
0811 06:32:55.617147 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0811 06:32:55.617222 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.193:37463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260811 06:32:55.689963 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 17915
I20260811 06:32:55.715637 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 17348
I20260811 06:32:55.716950 17447 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:32:55.834659 18205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:32:55.833945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.193:37463 (local address 127.0.0.1:56124)
0811 06:32:55.834280 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0811 06:32:55.834418 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.193:37463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260811 06:32:55.834908 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 17348
I20260811 06:32:55.873684 29631 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 06:32:56.219816 29631 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 06:32:56.220141 29631 mini_ranger.cc:66] Stopping Ranger...
I20260811 06:32:56.566632 29631 mini_ranger.cc:68] Stopped Ranger
2026-08-11 06:32:56.566 UTC [15894] LOG: received smart shutdown request
2026-08-11 06:32:56.571 UTC [15894] LOG: background worker "logical replication launcher" (PID 15903) exited with exit code 1
2026-08-11 06:32:56.571 UTC [15898] LOG: shutting down
2026-08-11 06:32:56.759 UTC [15894] LOG: database system is shut down
2026-08-11T06:32:56Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (56655 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429755927508-29631-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:32:57 dist-test-slave-gkw7 krb5kdc[18223](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:32:57 dist-test-slave-gkw7 krb5kdc[18223](info): set up 2 sockets
Aug 11 06:32:57 dist-test-slave-gkw7 krb5kdc[18223](info): commencing operation
krb5kdc: starting...
W20260811 06:32:58.294159 29631 mini_kdc.cc:121] Time spent starting KDC: real 1.352s user 0.001s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429755927508-29631-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429755927508-29631-0/krb5kdc/test-user.keytab.
Aug 11 06:32:58 dist-test-slave-gkw7 krb5kdc[18223](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429978, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:32:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:32:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.239.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.239.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.239.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab.
Entry for principal kudu/127.28.239.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab.
I20260811 06:32:58.606279 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429755927508-29631-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429755927508-29631-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429755927508-29631-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:34603
--webserver_interface=127.28.239.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab
--principal=kudu/127.28.239.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:40249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.239.254:34603,127.28.239.253:42215,127.28.239.252:38959
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:32:58.965471 18244 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:32:58.965893 18244 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:32:58.965991 18244 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:32:58.973384 18244 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:32:58.973513 18244 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:32:58.973654 18244 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:32:58.973724 18244 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:32:58.973773 18244 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:32:58.984308 18244 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:40249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429755927508-29631-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429755927508-29631-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.239.254:34603,127.28.239.253:42215,127.28.239.252:38959
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab
--principal=kudu/127.28.239.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:34603
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.239.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429755927508-29631-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:32:58.985711 18244 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:32:58.987102 18244 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:32:58.999740 18253 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:32:58.999827 18254 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:32:58.999791 18256 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:32:59.000797 18244 server_base.cc:1034] running on GCE node
I20260811 06:32:59.001717 18244 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:32:59.003222 18244 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:32:59.004480 18244 hybrid_clock.cc:648] HybridClock initialized: now 1786429979004433 us; error 41 us; skew 500 ppm
I20260811 06:32:59.004927 18244 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:32:59 dist-test-slave-gkw7 krb5kdc[18223](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429979, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:32:59.013963 18244 init.cc:377] Logged in from keytab as kudu/127.28.239.254@KRBTEST.COM (short username kudu)
I20260811 06:32:59.015774 18244 webserver.cc:460] Webserver started at http://127.28.239.254:36281/ using document root <none> and password file <none>
I20260811 06:32:59.016512 18244 fs_manager.cc:359] Metadata directory not provided
I20260811 06:32:59.016619 18244 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:32:59.016896 18244 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:32:59.020118 18244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429755927508-29631-0/minicluster-data/master-0/data/instance:
uuid: "3a065545ac85450089dd1b7ed59e03fe"
format_stamp: "Formatted at 2026-08-11 06:32:59 on dist-test-slave-gkw7"
server_key: "66eb4aa9626e3fe7dc61fdb56ec262ce"
server_key_iv: "e9f7f00840beb29a8d8730642ecc6f07"
server_key_version: "encryptionkey@0"
I20260811 06:32:59.020916 18244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429755927508-29631-0/minicluster-data/master-0/wal/instance:
uuid: "3a065545ac85450089dd1b7ed59e03fe"
format_stamp: "Formatted at 2026-08-11 06:32:59 on dist-test-slave-gkw7"
server_key: "66eb4aa9626e3fe7dc61fdb56ec262ce"
server_key_iv: "e9f7f00840beb29a8d8730642ecc6f07"
server_key_version: "encryptionkey@0"
I20260811 06:32:59.026608 18244 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260811 06:32:59.030331 18267 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:32:59.031728 18244 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 06:32:59.031905 18244 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429755927508-29631-0/minicluster-data/master-0/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429755927508-29631-0/minicluster-data/master-0/wal
uuid: "3a065545ac85450089dd1b7ed59e03fe"
format_stamp: "Formatted at 2026-08-11 06:32:59 on dist-test-slave-gkw7"
server_key: "66eb4aa9626e3fe7dc61fdb56ec262ce"
server_key_iv: "e9f7f00840beb29a8d8730642ecc6f07"
server_key_version: "encryptionkey@0"
I20260811 06:32:59.032106 18244 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429755927508-29631-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429755927508-29631-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429755927508-29631-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:32:59.059775 18244 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:32:59.064556 18244 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:32:59.064934 18244 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:32:59.086966 18244 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.254:34603
I20260811 06:32:59.086988 18356 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.254:34603 every 8 connection(s)
I20260811 06:32:59.088582 18244 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
I20260811 06:32:59.092593 18357 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:32:59.098321 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 18244
I20260811 06:32:59.098573 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429755927508-29631-0/minicluster-data/master-0/wal/instance
I20260811 06:32:59.099252 29631 external_mini_cluster.cc:1428] Setting key 4cc16083484415cdf64bd79f44e848e4
I20260811 06:32:59.100871 18357 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.239.254" port: 34603 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 06:32:59 dist-test-slave-gkw7 krb5kdc[18223](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429979, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.254@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.28.239.253@KRBTEST.COM; defaulting to no policy
I20260811 06:32:59.124835 18365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:32:59.104392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:34603 (local address 127.0.0.1:57310)
0811 06:32:59.105765 (+ 1373us) negotiation.cc:107] Waiting for socket to connect
0811 06:32:59.105806 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 06:32:59.107646 (+ 1840us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:32:59.108485 (+ 839us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:32:59.108491 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:32:59.109256 (+ 765us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:32:59.109662 (+ 406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:32:59.109682 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:32:59.111235 (+ 1553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:32:59.111241 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:32:59.112321 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:32:59.112332 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:32:59.113269 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:32:59.113277 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:32:59.113453 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:32:59.114213 (+ 760us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:32:59.114236 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:32:59.116733 (+ 2497us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:32:59.120206 (+ 3473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:32:59.120221 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:32:59.120234 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:32:59.120661 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:32:59.121109 (+ 448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:32:59.121114 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:32:59.121117 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:32:59.121336 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:32:59.122228 (+ 892us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:32:59.122241 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:32:59.122462 (+ 221us) client_negotiation.cc:769] Sending connection context
0811 06:32:59.122728 (+ 266us) client_negotiation.cc:240] Negotiation successful
0811 06:32:59.123375 (+ 647us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1112,"mutex_wait_us":158,"spinlock_wait_cycles":28928,"thread_start_us":913,"threads_started":1}
I20260811 06:32:59.124871 18362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:32:59.104378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57310 (local address 127.28.239.254:34603)
0811 06:32:59.105314 (+ 936us) server_negotiation.cc:206] Beginning negotiation
0811 06:32:59.105333 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 06:32:59.105931 (+ 598us) server_negotiation.cc:407] Connection header received
0811 06:32:59.108020 (+ 2089us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:32:59.108025 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:32:59.108263 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:32:59.108472 (+ 209us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:32:59.109863 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:32:59.111071 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:32:59.112464 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:32:59.113018 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:32:59.113236 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:32:59.116976 (+ 3740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:32:59.116984 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:32:59.116996 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:32:59.117026 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:32:59.120048 (+ 3022us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:32:59.120824 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:32:59.120831 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:32:59.120834 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:32:59.120981 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:32:59.121460 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:32:59.121471 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:32:59.121473 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:32:59.122078 (+ 605us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:32:59.122313 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0811 06:32:59.122870 (+ 557us) server_negotiation.cc:299] Negotiation successful
0811 06:32:59.123257 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":26,"server-negotiator.queue_time_us":683,"thread_start_us":451,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.239.253@KRBTEST.COM" created.
I20260811 06:32:59.128003 18357 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.239.253" port: 42215 } has no permanent_uuid. Determining permanent_uuid...
I20260811 06:32:59.129029 18365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:32:59.128625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.253:42215 (local address 127.0.0.1:34150)
0811 06:32:59.128794 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0811 06:32:59.128916 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.253:42215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260811 06:32:59.129328 18278 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.28.239.253:42215: connect: Connection refused (error 111)
I20260811 06:32:59.131145 18365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:32:59.130699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.253:42215 (local address 127.0.0.1:34158)
0811 06:32:59.130917 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0811 06:32:59.130996 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.253:42215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260811 06:32:59.131394 18357 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.239.253:42215: Network error: Client connection negotiation failed: client connection to 127.28.239.253:42215: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.239.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.253.keytab.
Entry for principal kudu/127.28.239.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.253.keytab.
I20260811 06:32:59.161818 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429755927508-29631-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429755927508-29631-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429755927508-29631-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429755927508-29631-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.253:42215
--webserver_interface=127.28.239.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.253.keytab
--principal=kudu/127.28.239.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:40249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.239.254:34603,127.28.239.253:42215,127.28.239.252:38959
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 06:32:59.180860 18357 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.239.253" port: 42215 } attempt: 1
I20260811 06:32:59.182396 18365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:32:59.181822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.253:42215 (local address 127.0.0.1:34164)
0811 06:32:59.182057 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0811 06:32:59.182243 (+ 186us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.253:42215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260811 06:32:59.183559 18365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:32:59.183168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.253:42215 (local address 127.0.0.1:34172)
0811 06:32:59.183348 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0811 06:32:59.183422 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.253:42215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260811 06:32:59.183830 18357 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.239.253:42215: Network error: Client connection negotiation failed: client connection to 127.28.239.253:42215: connect: Connection refused (error 111)
I20260811 06:32:59.252156 18357 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.239.253" port: 42215 } attempt: 2
I20260811 06:32:59.253582 18365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:32:59.252935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.253:42215 (local address 127.0.0.1:34174)
0811 06:32:59.253232 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0811 06:32:59.253369 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.253:42215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260811 06:32:59.254748 18365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:32:59.254292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.253:42215 (local address 127.0.0.1:34182)
0811 06:32:59.254489 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0811 06:32:59.254577 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.253:42215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
W20260811 06:32:59.254961 18357 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.239.253:42215: Network error: Client connection negotiation failed: client connection to 127.28.239.253:42215: connect: Connection refused (error 111)
I20260811 06:32:59.346282 18357 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.239.253" port: 42215 } attempt: 3
I20260811 06:32:59.347586 18365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:32:59.347044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.253:42215 (local address 127.0.0.1:34196)
0811 06:32:59.347294 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0811 06:32:59.347412 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.253:42215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260811 06:32:59.348944 18365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:32:59.348489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.253:42215 (local address 127.0.0.1:34198)
0811 06:32:59.348663 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0811 06:32:59.348775 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.253:42215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260811 06:32:59.349234 18357 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.239.253:42215: Network error: Client connection negotiation failed: client connection to 127.28.239.253:42215: connect: Connection refused (error 111)
I20260811 06:32:59.492533 18357 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.239.253" port: 42215 } attempt: 4
I20260811 06:32:59.493997 18365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:32:59.493278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.253:42215 (local address 127.0.0.1:34202)
0811 06:32:59.493695 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0811 06:32:59.493818 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.253:42215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188}
I20260811 06:32:59.495150 18365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:32:59.494739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.253:42215 (local address 127.0.0.1:34212)
0811 06:32:59.494903 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0811 06:32:59.495004 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.253:42215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260811 06:32:59.495355 18357 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.239.253:42215: Network error: Client connection negotiation failed: client connection to 127.28.239.253:42215: connect: Connection refused (error 111)
W20260811 06:32:59.514846 18369 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:32:59.515139 18369 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:32:59.515305 18369 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:32:59.522646 18369 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:32:59.522770 18369 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:32:59.522902 18369 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:32:59.523000 18369 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:32:59.523087 18369 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:32:59.533788 18369 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:40249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429755927508-29631-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429755927508-29631-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.28.239.254:34603,127.28.239.253:42215,127.28.239.252:38959
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.253.keytab
--principal=kudu/127.28.239.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.253:42215
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429755927508-29631-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.28.239.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429755927508-29631-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:32:59.535243 18369 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:32:59.536718 18369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:32:59.549018 18380 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:32:59.548982 18383 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:32:59.549065 18381 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:32:59.550331 18369 server_base.cc:1034] running on GCE node
I20260811 06:32:59.551136 18369 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:32:59.552630 18369 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:32:59.553860 18369 hybrid_clock.cc:648] HybridClock initialized: now 1786429979553793 us; error 68 us; skew 500 ppm
I20260811 06:32:59.554224 18369 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:32:59 dist-test-slave-gkw7 krb5kdc[18223](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429979, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:32:59.557708 18369 init.cc:377] Logged in from keytab as kudu/127.28.239.253@KRBTEST.COM (short username kudu)
I20260811 06:32:59.559389 18369 webserver.cc:460] Webserver started at http://127.28.239.253:42667/ using document root <none> and password file <none>
I20260811 06:32:59.560135 18369 fs_manager.cc:359] Metadata directory not provided
I20260811 06:32:59.560289 18369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:32:59.560617 18369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:32:59.563808 18369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429755927508-29631-0/minicluster-data/master-1/data/instance:
uuid: "205eaac6700e484aa2429c74bf9d5f97"
format_stamp: "Formatted at 2026-08-11 06:32:59 on dist-test-slave-gkw7"
server_key: "fbd52488b6753657ee96fccc01f30eab"
server_key_iv: "2239d6ef0315c0c8c7c4ab47b1e956d2"
server_key_version: "encryptionkey@0"
I20260811 06:32:59.564644 18369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429755927508-29631-0/minicluster-data/master-1/wal/instance:
uuid: "205eaac6700e484aa2429c74bf9d5f97"
format_stamp: "Formatted at 2026-08-11 06:32:59 on dist-test-slave-gkw7"
server_key: "fbd52488b6753657ee96fccc01f30eab"
server_key_iv: "2239d6ef0315c0c8c7c4ab47b1e956d2"
server_key_version: "encryptionkey@0"
I20260811 06:32:59.570168 18369 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 06:32:59.574156 18394 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:32:59.575574 18369 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 06:32:59.575766 18369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429755927508-29631-0/minicluster-data/master-1/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429755927508-29631-0/minicluster-data/master-1/wal
uuid: "205eaac6700e484aa2429c74bf9d5f97"
format_stamp: "Formatted at 2026-08-11 06:32:59 on dist-test-slave-gkw7"
server_key: "fbd52488b6753657ee96fccc01f30eab"
server_key_iv: "2239d6ef0315c0c8c7c4ab47b1e956d2"
server_key_version: "encryptionkey@0"
I20260811 06:32:59.575947 18369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429755927508-29631-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429755927508-29631-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429755927508-29631-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:32:59.598480 18369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:32:59.603201 18369 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:32:59.603539 18369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:32:59.622299 18369 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.253:42215
I20260811 06:32:59.622323 18471 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.253:42215 every 8 connection(s)
I20260811 06:32:59.623842 18369 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429755927508-29631-0/minicluster-data/master-1/data/info.pb
I20260811 06:32:59.627799 18477 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:32:59.632247 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 18369
I20260811 06:32:59.632441 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429755927508-29631-0/minicluster-data/master-1/wal/instance
I20260811 06:32:59.632899 29631 external_mini_cluster.cc:1428] Setting key d1ff0ea29c5f1c7dc4bcd6e62bd92481
I20260811 06:32:59.634898 18477 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.239.254" port: 34603 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 06:32:59 dist-test-slave-gkw7 krb5kdc[18223](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429979, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.253@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:32:59.659531 18484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:32:59.637786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57314 (local address 127.28.239.254:34603)
0811 06:32:59.638337 (+ 551us) server_negotiation.cc:206] Beginning negotiation
0811 06:32:59.638343 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:32:59.639589 (+ 1246us) server_negotiation.cc:407] Connection header received
0811 06:32:59.642925 (+ 3336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:32:59.642934 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:32:59.643043 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:32:59.643164 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:32:59.645475 (+ 2311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:32:59.646269 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:32:59.647922 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:32:59.648353 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:32:59.648494 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:32:59.652204 (+ 3710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:32:59.652210 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:32:59.652213 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:32:59.652259 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:32:59.655385 (+ 3126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:32:59.656443 (+ 1058us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:32:59.656450 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:32:59.656455 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:32:59.656625 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:32:59.657197 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:32:59.657205 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:32:59.657211 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:32:59.657711 (+ 500us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:32:59.657899 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0811 06:32:59.658787 (+ 888us) server_negotiation.cc:299] Negotiation successful
0811 06:32:59.659061 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":278,"threads_started":1}
I20260811 06:32:59.660487 18485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:32:59.638465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:34603 (local address 127.0.0.1:57314)
0811 06:32:59.639373 (+ 908us) negotiation.cc:107] Waiting for socket to connect
0811 06:32:59.639437 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0811 06:32:59.642538 (+ 3101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:32:59.643509 (+ 971us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:32:59.643523 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:32:59.644400 (+ 877us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:32:59.645247 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:32:59.645269 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:32:59.646459 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:32:59.646463 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:32:59.647740 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:32:59.647753 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:32:59.648611 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:32:59.648619 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:32:59.648862 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:32:59.649678 (+ 816us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:32:59.649702 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:32:59.651946 (+ 2244us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:32:59.655596 (+ 3650us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:32:59.655607 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:32:59.655628 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:32:59.656258 (+ 630us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:32:59.656774 (+ 516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:32:59.656779 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:32:59.656782 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:32:59.657009 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:32:59.657879 (+ 870us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:32:59.657887 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:32:59.658366 (+ 479us) client_negotiation.cc:769] Sending connection context
0811 06:32:59.658655 (+ 289us) client_negotiation.cc:240] Negotiation successful
0811 06:32:59.659105 (+ 450us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":558,"thread_start_us":325,"threads_started":1}
WARNING: no policy specified for kudu/127.28.239.252@KRBTEST.COM; defaulting to no policy
I20260811 06:32:59.662598 18477 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.239.253" port: 42215 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.239.252@KRBTEST.COM" created.
Aug 11 06:32:59 dist-test-slave-gkw7 krb5kdc[18223](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429979, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.253@KRBTEST.COM for kudu/127.28.239.253@KRBTEST.COM
I20260811 06:32:59.679529 18485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:32:59.663465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.253:42215 (local address 127.0.0.1:34226)
0811 06:32:59.663661 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0811 06:32:59.663682 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 06:32:59.663805 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:32:59.664863 (+ 1058us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:32:59.664875 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:32:59.665409 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:32:59.665653 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:32:59.665671 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:32:59.667279 (+ 1608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:32:59.667287 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:32:59.668778 (+ 1491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:32:59.668796 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:32:59.669817 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:32:59.669825 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:32:59.670068 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:32:59.670769 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:32:59.670797 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:32:59.673577 (+ 2780us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:32:59.676889 (+ 3312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:32:59.676897 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:32:59.676901 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:32:59.677267 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:32:59.677692 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:32:59.677698 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:32:59.677702 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:32:59.677863 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:32:59.678501 (+ 638us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:32:59.678505 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:32:59.678670 (+ 165us) client_negotiation.cc:769] Sending connection context
0811 06:32:59.678817 (+ 147us) client_negotiation.cc:240] Negotiation successful
0811 06:32:59.679133 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72}
I20260811 06:32:59.679661 18488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:32:59.663758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34226 (local address 127.28.239.253:42215)
0811 06:32:59.664405 (+ 647us) server_negotiation.cc:206] Beginning negotiation
0811 06:32:59.664412 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:32:59.664441 (+ 29us) server_negotiation.cc:407] Connection header received
0811 06:32:59.664487 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:32:59.664493 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:32:59.664684 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:32:59.664814 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:32:59.665859 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:32:59.667113 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:32:59.668954 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:32:59.669656 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:32:59.669768 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:32:59.673823 (+ 4055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:32:59.673835 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:32:59.673849 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:32:59.673877 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:32:59.676750 (+ 2873us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:32:59.677403 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:32:59.677416 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:32:59.677418 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:32:59.677577 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:32:59.677969 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:32:59.677976 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:32:59.677979 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:32:59.678358 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:32:59.678583 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0811 06:32:59.678976 (+ 393us) server_negotiation.cc:299] Negotiation successful
0811 06:32:59.679259 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":40,"server-negotiator.queue_time_us":490,"thread_start_us":324,"threads_started":1}
I20260811 06:32:59.682363 18477 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.239.252" port: 38959 } has no permanent_uuid. Determining permanent_uuid...
I20260811 06:32:59.683601 18485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:32:59.683167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.252:38959 (local address 127.0.0.1:53058)
0811 06:32:59.683331 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0811 06:32:59.683450 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.252:38959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260811 06:32:59.683923 18395 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.28.239.252:38959: connect: Connection refused (error 111)
I20260811 06:32:59.685670 18485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:32:59.685251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.252:38959 (local address 127.0.0.1:53064)
0811 06:32:59.685435 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0811 06:32:59.685510 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.252:38959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260811 06:32:59.685873 18477 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.239.252:38959: Network error: Client connection negotiation failed: client connection to 127.28.239.252:38959: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.239.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.252.keytab.
Entry for principal kudu/127.28.239.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.252.keytab.
I20260811 06:32:59.702244 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429755927508-29631-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429755927508-29631-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429755927508-29631-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429755927508-29631-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.252:38959
--webserver_interface=127.28.239.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.252.keytab
--principal=kudu/127.28.239.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:40249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.239.254:34603,127.28.239.253:42215,127.28.239.252:38959
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 06:32:59.735316 18477 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.239.252" port: 38959 } attempt: 1
I20260811 06:32:59.736732 18485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:32:59.736153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.252:38959 (local address 127.0.0.1:53066)
0811 06:32:59.736389 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0811 06:32:59.736538 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.252:38959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260811 06:32:59.737872 18485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:32:59.737477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.252:38959 (local address 127.0.0.1:53070)
0811 06:32:59.737687 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0811 06:32:59.737766 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.252:38959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
W20260811 06:32:59.738148 18477 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.239.252:38959: Network error: Client connection negotiation failed: client connection to 127.28.239.252:38959: connect: Connection refused (error 111)
I20260811 06:32:59.794704 18357 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.239.253" port: 42215 } attempt: 5
Aug 11 06:32:59 dist-test-slave-gkw7 krb5kdc[18223](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429979, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.254@KRBTEST.COM for kudu/127.28.239.253@KRBTEST.COM
I20260811 06:32:59.806397 18477 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.239.252" port: 38959 } attempt: 2
I20260811 06:32:59.807629 18485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:32:59.807066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.252:38959 (local address 127.0.0.1:53074)
0811 06:32:59.807278 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0811 06:32:59.807384 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.252:38959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260811 06:32:59.808815 18485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:32:59.808386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.252:38959 (local address 127.0.0.1:53082)
0811 06:32:59.808567 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0811 06:32:59.808654 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.252:38959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260811 06:32:59.809139 18477 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.239.252:38959: Network error: Client connection negotiation failed: client connection to 127.28.239.252:38959: connect: Connection refused (error 111)
I20260811 06:32:59.811112 18488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:32:59.795763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34232 (local address 127.28.239.253:42215)
0811 06:32:59.795988 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 06:32:59.795996 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:32:59.796024 (+ 28us) server_negotiation.cc:407] Connection header received
0811 06:32:59.796140 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:32:59.796148 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:32:59.796247 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:32:59.796378 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:32:59.797241 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:32:59.798393 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:32:59.799673 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:32:59.800314 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:32:59.800445 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:32:59.804195 (+ 3750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:32:59.804202 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:32:59.804207 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:32:59.804249 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:32:59.808087 (+ 3838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:32:59.809029 (+ 942us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:32:59.809034 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:32:59.809039 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:32:59.809207 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:32:59.809653 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:32:59.809658 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:32:59.809662 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:32:59.809973 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:32:59.810135 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 06:32:59.810563 (+ 428us) server_negotiation.cc:299] Negotiation successful
0811 06:32:59.810800 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 06:32:59.811151 18365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:32:59.795596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.253:42215 (local address 127.0.0.1:34232)
0811 06:32:59.795839 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0811 06:32:59.795885 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 06:32:59.796008 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:32:59.796431 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:32:59.796435 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:32:59.796843 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:32:59.797055 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:32:59.797064 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:32:59.798547 (+ 1483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:32:59.798552 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:32:59.799513 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:32:59.799525 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:32:59.800474 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:32:59.800479 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:32:59.800678 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:32:59.801420 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:32:59.801439 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:32:59.803969 (+ 2530us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:32:59.808361 (+ 4392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:32:59.808367 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:32:59.808371 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:32:59.808842 (+ 471us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:32:59.809343 (+ 501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:32:59.809348 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:32:59.809352 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:32:59.809507 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:32:59.810104 (+ 597us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:32:59.810109 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:32:59.810293 (+ 184us) client_negotiation.cc:769] Sending connection context
0811 06:32:59.810448 (+ 155us) client_negotiation.cc:240] Negotiation successful
0811 06:32:59.810799 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83}
I20260811 06:32:59.812559 18357 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.239.252" port: 38959 } has no permanent_uuid. Determining permanent_uuid...
I20260811 06:32:59.813791 18365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:32:59.813335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.252:38959 (local address 127.0.0.1:53098)
0811 06:32:59.813503 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0811 06:32:59.813636 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.252:38959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260811 06:32:59.814852 18365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:32:59.814451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.252:38959 (local address 127.0.0.1:53108)
0811 06:32:59.814596 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0811 06:32:59.814728 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.252:38959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260811 06:32:59.815049 18357 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.239.252:38959: Network error: Client connection negotiation failed: client connection to 127.28.239.252:38959: connect: Connection refused (error 111)
I20260811 06:32:59.866312 18357 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.239.252" port: 38959 } attempt: 1
I20260811 06:32:59.867554 18365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:32:59.867038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.252:38959 (local address 127.0.0.1:53110)
0811 06:32:59.867268 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0811 06:32:59.867375 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.252:38959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260811 06:32:59.868592 18365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:32:59.868261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.252:38959 (local address 127.0.0.1:53122)
0811 06:32:59.868416 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0811 06:32:59.868491 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.252:38959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260811 06:32:59.868814 18357 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.239.252:38959: Network error: Client connection negotiation failed: client connection to 127.28.239.252:38959: connect: Connection refused (error 111)
I20260811 06:32:59.900375 18477 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.239.252" port: 38959 } attempt: 3
I20260811 06:32:59.901935 18485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:32:59.901214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.252:38959 (local address 127.0.0.1:53128)
0811 06:32:59.901497 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0811 06:32:59.901665 (+ 168us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.252:38959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260811 06:32:59.903256 18485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:32:59.902730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.252:38959 (local address 127.0.0.1:53140)
0811 06:32:59.903001 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0811 06:32:59.903097 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.252:38959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158}
W20260811 06:32:59.903512 18477 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.239.252:38959: Network error: Client connection negotiation failed: client connection to 127.28.239.252:38959: connect: Connection refused (error 111)
I20260811 06:32:59.937114 18357 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.239.252" port: 38959 } attempt: 2
I20260811 06:32:59.938628 18365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:32:59.937994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.252:38959 (local address 127.0.0.1:53142)
0811 06:32:59.938281 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0811 06:32:59.938402 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.252:38959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260811 06:32:59.939846 18365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:32:59.939425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.252:38959 (local address 127.0.0.1:53146)
0811 06:32:59.939609 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0811 06:32:59.939705 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.252:38959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260811 06:32:59.940150 18357 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.239.252:38959: Network error: Client connection negotiation failed: client connection to 127.28.239.252:38959: connect: Connection refused (error 111)
I20260811 06:33:00.046396 18357 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.239.252" port: 38959 } attempt: 3
I20260811 06:33:00.046784 18477 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.239.252" port: 38959 } attempt: 4
I20260811 06:33:00.047752 18365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:00.047150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.252:38959 (local address 127.0.0.1:53150)
0811 06:33:00.047388 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:00.047514 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.252:38959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
I20260811 06:33:00.047968 18485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:00.047427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.252:38959 (local address 127.0.0.1:53164)
0811 06:33:00.047674 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:00.047780 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.252:38959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260811 06:33:00.048978 18365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:00.048538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.252:38959 (local address 127.0.0.1:53168)
0811 06:33:00.048745 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:00.048846 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.252:38959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
W20260811 06:33:00.049160 18357 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.239.252:38959: Network error: Client connection negotiation failed: client connection to 127.28.239.252:38959: connect: Connection refused (error 111)
I20260811 06:33:00.049306 18485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:00.048734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.252:38959 (local address 127.0.0.1:53174)
0811 06:33:00.048898 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:00.048979 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.252:38959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260811 06:33:00.049517 18477 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.239.252:38959: Network error: Client connection negotiation failed: client connection to 127.28.239.252:38959: connect: Connection refused (error 111)
W20260811 06:33:00.062422 18493 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:33:00.062740 18493 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:33:00.062947 18493 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:33:00.071473 18493 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:33:00.071604 18493 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:33:00.071708 18493 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:33:00.071812 18493 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:33:00.071898 18493 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:33:00.082780 18493 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:40249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429755927508-29631-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429755927508-29631-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.28.239.254:34603,127.28.239.253:42215,127.28.239.252:38959
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.252.keytab
--principal=kudu/127.28.239.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.252:38959
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429755927508-29631-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.28.239.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429755927508-29631-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:33:00.084364 18493 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:33:00.085870 18493 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:33:00.099767 18501 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:33:00.099812 18498 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:33:00.100912 18493 server_base.cc:1034] running on GCE node
W20260811 06:33:00.101432 18507 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:33:00.102448 18493 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:33:00.104009 18493 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:33:00.105263 18493 hybrid_clock.cc:648] HybridClock initialized: now 1786429980105221 us; error 94 us; skew 500 ppm
I20260811 06:33:00.105726 18493 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:33:00 dist-test-slave-gkw7 krb5kdc[18223](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429980, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:33:00.109180 18493 init.cc:377] Logged in from keytab as kudu/127.28.239.252@KRBTEST.COM (short username kudu)
I20260811 06:33:00.110767 18493 webserver.cc:460] Webserver started at http://127.28.239.252:33263/ using document root <none> and password file <none>
I20260811 06:33:00.111460 18493 fs_manager.cc:359] Metadata directory not provided
I20260811 06:33:00.111562 18493 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:33:00.111824 18493 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:33:00.114908 18493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429755927508-29631-0/minicluster-data/master-2/data/instance:
uuid: "892ce2df978547d0b50aee4db0626860"
format_stamp: "Formatted at 2026-08-11 06:33:00 on dist-test-slave-gkw7"
server_key: "1499cff712e35759d776409ad1b2bbec"
server_key_iv: "9e106817ba087a77e1c0132e08882041"
server_key_version: "encryptionkey@0"
I20260811 06:33:00.115692 18493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429755927508-29631-0/minicluster-data/master-2/wal/instance:
uuid: "892ce2df978547d0b50aee4db0626860"
format_stamp: "Formatted at 2026-08-11 06:33:00 on dist-test-slave-gkw7"
server_key: "1499cff712e35759d776409ad1b2bbec"
server_key_iv: "9e106817ba087a77e1c0132e08882041"
server_key_version: "encryptionkey@0"
I20260811 06:33:00.121076 18493 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 06:33:00.124893 18516 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:33:00.126493 18493 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 06:33:00.126652 18493 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429755927508-29631-0/minicluster-data/master-2/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429755927508-29631-0/minicluster-data/master-2/wal
uuid: "892ce2df978547d0b50aee4db0626860"
format_stamp: "Formatted at 2026-08-11 06:33:00 on dist-test-slave-gkw7"
server_key: "1499cff712e35759d776409ad1b2bbec"
server_key_iv: "9e106817ba087a77e1c0132e08882041"
server_key_version: "encryptionkey@0"
I20260811 06:33:00.126837 18493 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429755927508-29631-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429755927508-29631-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429755927508-29631-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:33:00.148079 18493 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:33:00.152678 18493 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:33:00.152940 18493 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:33:00.171528 18493 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.252:38959
I20260811 06:33:00.171540 18598 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.252:38959 every 8 connection(s)
I20260811 06:33:00.173018 18493 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429755927508-29631-0/minicluster-data/master-2/data/info.pb
I20260811 06:33:00.176873 18599 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:33:00.182387 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 18493
I20260811 06:33:00.182616 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786429755927508-29631-0/minicluster-data/master-2/wal/instance
I20260811 06:33:00.183109 29631 external_mini_cluster.cc:1428] Setting key 3eb3e5dd38c97d73fd5c6ab0fb9891c6
I20260811 06:33:00.184253 18599 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.239.254" port: 34603 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 06:33:00 dist-test-slave-gkw7 krb5kdc[18223](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429980, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.252@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
Aug 11 06:33:00 dist-test-slave-gkw7 krb5kdc[18223](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429978, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.239.252@KRBTEST.COM
I20260811 06:33:00.206338 18606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:00.188059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57324 (local address 127.28.239.254:34603)
0811 06:33:00.189431 (+ 1372us) server_negotiation.cc:206] Beginning negotiation
0811 06:33:00.189437 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:33:00.189460 (+ 23us) server_negotiation.cc:407] Connection header received
0811 06:33:00.189746 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:33:00.189751 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:33:00.189873 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:33:00.189995 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:33:00.191329 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:00.192112 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:00.193979 (+ 1867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:00.194737 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:00.194907 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:00.199469 (+ 4562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:33:00.199479 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:33:00.199487 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:33:00.199547 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:33:00.202962 (+ 3415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:33:00.203796 (+ 834us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:33:00.203801 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:33:00.203803 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:33:00.203940 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:33:00.204442 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:33:00.204453 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:33:00.204455 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:33:00.204746 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:33:00.204932 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0811 06:33:00.205643 (+ 711us) server_negotiation.cc:299] Negotiation successful
0811 06:33:00.205893 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1240,"thread_start_us":1106,"threads_started":1}
I20260811 06:33:00.207396 18602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:00.185191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53188 (local address 127.28.239.252:38959)
0811 06:33:00.186572 (+ 1381us) server_negotiation.cc:206] Beginning negotiation
0811 06:33:00.186588 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 06:33:00.186642 (+ 54us) server_negotiation.cc:407] Connection header received
0811 06:33:00.189382 (+ 2740us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:33:00.189405 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:33:00.189716 (+ 311us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:33:00.189883 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:33:00.191471 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:00.192981 (+ 1510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:00.194987 (+ 2006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:00.195787 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:00.195925 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:00.200398 (+ 4473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:33:00.200408 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:33:00.200422 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:33:00.200453 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:33:00.203293 (+ 2840us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:33:00.204020 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:33:00.204028 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:33:00.204030 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:33:00.204193 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:33:00.204638 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:33:00.204643 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:33:00.204645 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:33:00.205234 (+ 589us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:33:00.205543 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0811 06:33:00.205927 (+ 384us) server_negotiation.cc:299] Negotiation successful
0811 06:33:00.206230 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":42,"server-negotiator.queue_time_us":1150,"spinlock_wait_cycles":15616,"thread_start_us":943,"threads_started":1}
I20260811 06:33:00.207458 18605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:00.188007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:34603 (local address 127.0.0.1:57324)
0811 06:33:00.188749 (+ 742us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:00.188800 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 06:33:00.189404 (+ 604us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:33:00.190091 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:33:00.190098 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:33:00.190757 (+ 659us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:33:00.191133 (+ 376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:33:00.191150 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:33:00.192291 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:33:00.192296 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:33:00.193831 (+ 1535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:33:00.193843 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:33:00.194989 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:33:00.194995 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:33:00.195273 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:00.196396 (+ 1123us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:33:00.196454 (+ 58us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:33:00.199185 (+ 2731us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:33:00.203146 (+ 3961us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:33:00.203159 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:33:00.203173 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:33:00.203641 (+ 468us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:33:00.204082 (+ 441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:33:00.204088 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:33:00.204093 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:33:00.204300 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:33:00.204891 (+ 591us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:33:00.204901 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:33:00.205204 (+ 303us) client_negotiation.cc:769] Sending connection context
0811 06:33:00.205483 (+ 279us) client_negotiation.cc:240] Negotiation successful
0811 06:33:00.206113 (+ 630us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":326,"threads_started":1}
I20260811 06:33:00.210155 18599 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.239.253" port: 42215 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 06:33:00 dist-test-slave-gkw7 krb5kdc[18223](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429980, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.252@KRBTEST.COM for kudu/127.28.239.253@KRBTEST.COM
Aug 11 06:33:00 dist-test-slave-gkw7 krb5kdc[18223](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429978, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:33:00.225311 18605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:00.210827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.253:42215 (local address 127.0.0.1:34244)
0811 06:33:00.211116 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:00.211138 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 06:33:00.211261 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:33:00.211637 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:33:00.211642 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:33:00.212221 (+ 579us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:33:00.212474 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:33:00.212490 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:33:00.213631 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:33:00.213636 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:33:00.215039 (+ 1403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:33:00.215055 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:33:00.215789 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:33:00.215798 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:33:00.216013 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:00.216686 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:33:00.216715 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:33:00.219402 (+ 2687us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:33:00.222832 (+ 3430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:33:00.222838 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:33:00.222843 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:33:00.223219 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:33:00.223623 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:33:00.223629 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:33:00.223632 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:33:00.223793 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:33:00.224312 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:33:00.224317 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:33:00.224477 (+ 160us) client_negotiation.cc:769] Sending connection context
0811 06:33:00.224620 (+ 143us) client_negotiation.cc:240] Negotiation successful
0811 06:33:00.224921 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154}
I20260811 06:33:00.225332 18488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:00.210964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34244 (local address 127.28.239.253:42215)
0811 06:33:00.211140 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 06:33:00.211146 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:33:00.211213 (+ 67us) server_negotiation.cc:407] Connection header received
0811 06:33:00.211392 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:33:00.211397 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:33:00.211479 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:33:00.211582 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:33:00.212637 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:00.213406 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:00.215211 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:00.215636 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:00.215731 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:00.219598 (+ 3867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:33:00.219603 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:33:00.219606 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:33:00.219638 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:33:00.222676 (+ 3038us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:33:00.223354 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:33:00.223359 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:33:00.223365 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:33:00.223503 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:33:00.223907 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:33:00.223913 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:33:00.223916 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:33:00.224185 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:33:00.224354 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0811 06:33:00.224738 (+ 384us) server_negotiation.cc:299] Negotiation successful
0811 06:33:00.224962 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 06:33:00.226377 18599 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.239.252" port: 38959 } has no permanent_uuid. Determining permanent_uuid...
I20260811 06:33:00.226402 18357 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.239.252" port: 38959 } attempt: 4
I20260811 06:33:00.227437 18606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:00.210936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57336 (local address 127.28.239.254:34603)
0811 06:33:00.211109 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 06:33:00.211114 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:33:00.211218 (+ 104us) server_negotiation.cc:407] Connection header received
0811 06:33:00.211391 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:33:00.211396 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:33:00.211478 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:33:00.211581 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:33:00.213224 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:00.214083 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:00.215904 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:00.216426 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:00.216561 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:00.221588 (+ 5027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:33:00.221596 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:33:00.221602 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:33:00.221648 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:33:00.224532 (+ 2884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:33:00.225237 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:33:00.225242 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:33:00.225246 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:33:00.225422 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:33:00.225939 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:33:00.225943 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:33:00.225945 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:33:00.226263 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:33:00.226571 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0811 06:33:00.226857 (+ 286us) server_negotiation.cc:299] Negotiation successful
0811 06:33:00.227116 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
Aug 11 06:33:00 dist-test-slave-gkw7 krb5kdc[18223](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429979, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.254@KRBTEST.COM for kudu/127.28.239.252@KRBTEST.COM
Aug 11 06:33:00 dist-test-slave-gkw7 krb5kdc[18223](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429980, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.252@KRBTEST.COM for kudu/127.28.239.252@KRBTEST.COM
Aug 11 06:33:00 dist-test-slave-gkw7 krb5kdc[18223](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429978, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.239.253@KRBTEST.COM
I20260811 06:33:00.244599 18365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:00.227003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.252:38959 (local address 127.0.0.1:53196)
0811 06:33:00.227410 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:00.227435 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 06:33:00.227543 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:33:00.228476 (+ 933us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:33:00.228482 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:33:00.229032 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:33:00.229247 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:33:00.229262 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:33:00.230974 (+ 1712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:33:00.230982 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:33:00.232347 (+ 1365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:33:00.232361 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:33:00.233203 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:33:00.233209 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:33:00.233390 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:00.234087 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:33:00.234118 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:33:00.237467 (+ 3349us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:33:00.241575 (+ 4108us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:33:00.241585 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:33:00.241590 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:33:00.242182 (+ 592us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:33:00.242676 (+ 494us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:33:00.242683 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:33:00.242689 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:33:00.242877 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:33:00.243548 (+ 671us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:33:00.243554 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:33:00.243765 (+ 211us) client_negotiation.cc:769] Sending connection context
0811 06:33:00.243933 (+ 168us) client_negotiation.cc:240] Negotiation successful
0811 06:33:00.244228 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276}
I20260811 06:33:00.245867 18602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:00.227901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53196 (local address 127.28.239.252:38959)
0811 06:33:00.228099 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 06:33:00.228105 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:33:00.228122 (+ 17us) server_negotiation.cc:407] Connection header received
0811 06:33:00.228167 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:33:00.228170 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:33:00.228274 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:33:00.229399 (+ 1125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:33:00.229533 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:00.230819 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:00.232500 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:00.233030 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:00.233178 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:00.237726 (+ 4548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:33:00.237732 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:33:00.237735 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:33:00.237773 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:33:00.241376 (+ 3603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:33:00.242353 (+ 977us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:33:00.242361 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:33:00.242364 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:33:00.242529 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:33:00.242999 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:33:00.243004 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:33:00.243008 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:33:00.243412 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:33:00.243574 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 06:33:00.245117 (+ 1543us) server_negotiation.cc:299] Negotiation successful
0811 06:33:00.245412 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":26,"server-negotiator.queue_time_us":67,"spinlock_wait_cycles":1920}
I20260811 06:33:00.245954 18607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:00.227971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53210 (local address 127.28.239.252:38959)
0811 06:33:00.228639 (+ 668us) server_negotiation.cc:206] Beginning negotiation
0811 06:33:00.228646 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:33:00.228670 (+ 24us) server_negotiation.cc:407] Connection header received
0811 06:33:00.228714 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:33:00.228719 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:33:00.228776 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:33:00.228891 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:33:00.229907 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:00.231137 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:00.232692 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:00.233387 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:00.233504 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:00.238684 (+ 5180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:33:00.238691 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:33:00.238696 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:33:00.238733 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:33:00.242644 (+ 3911us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:33:00.243445 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:33:00.243453 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:33:00.243459 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:33:00.243610 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:33:00.243944 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:33:00.243951 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:33:00.243954 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:33:00.244296 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:33:00.244444 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 06:33:00.245346 (+ 902us) server_negotiation.cc:299] Negotiation successful
0811 06:33:00.245561 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":557,"thread_start_us":361,"threads_started":1}
I20260811 06:33:00.246963 18605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:00.227204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.252:38959 (local address 127.0.0.1:53210)
0811 06:33:00.227611 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:00.227631 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 06:33:00.227727 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:33:00.228946 (+ 1219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:33:00.228954 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:33:00.229439 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:33:00.229743 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:33:00.229757 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:33:00.231255 (+ 1498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:33:00.231262 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:33:00.232542 (+ 1280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:33:00.232556 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:33:00.234029 (+ 1473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:33:00.234036 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:33:00.234270 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:00.234920 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:33:00.234944 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:33:00.238407 (+ 3463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:33:00.242782 (+ 4375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:33:00.242789 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:33:00.242794 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:33:00.243313 (+ 519us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:33:00.243695 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:33:00.243702 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:33:00.243705 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:33:00.243835 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:33:00.244440 (+ 605us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:33:00.244446 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:33:00.244623 (+ 177us) client_negotiation.cc:769] Sending connection context
0811 06:33:00.245137 (+ 514us) client_negotiation.cc:240] Negotiation successful
0811 06:33:00.246519 (+ 1382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"mutex_wait_us":918}
I20260811 06:33:00.248447 18488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:00.229707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34260 (local address 127.28.239.253:42215)
0811 06:33:00.229889 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 06:33:00.229895 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:33:00.229910 (+ 15us) server_negotiation.cc:407] Connection header received
0811 06:33:00.229962 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:33:00.229965 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:33:00.230017 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:33:00.230136 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:33:00.231727 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:00.232798 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:00.234404 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:00.234992 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:00.235132 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:00.240438 (+ 5306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:33:00.240446 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:33:00.240451 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:33:00.240495 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:33:00.243938 (+ 3443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:33:00.244818 (+ 880us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:33:00.244823 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:33:00.244827 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:33:00.244985 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:33:00.245608 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:33:00.245614 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:33:00.245618 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:33:00.245965 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:33:00.246268 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0811 06:33:00.247887 (+ 1619us) server_negotiation.cc:299] Negotiation successful
0811 06:33:00.248113 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 06:33:00.252169 18357 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a065545ac85450089dd1b7ed59e03fe: Bootstrap starting.
I20260811 06:33:00.252406 18599 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 892ce2df978547d0b50aee4db0626860: Bootstrap starting.
I20260811 06:33:00.254370 29631 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 06:33:00.256048 18357 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3a065545ac85450089dd1b7ed59e03fe: Neither blocks nor log segments found. Creating new log.
I20260811 06:33:00.256145 18599 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 892ce2df978547d0b50aee4db0626860: Neither blocks nor log segments found. Creating new log.
I20260811 06:33:00.257191 18357 log.cc:824] T 00000000000000000000000000000000 P 3a065545ac85450089dd1b7ed59e03fe: Log is configured to *not* fsync() on all Append() calls
I20260811 06:33:00.257421 18599 log.cc:824] T 00000000000000000000000000000000 P 892ce2df978547d0b50aee4db0626860: Log is configured to *not* fsync() on all Append() calls
I20260811 06:33:00.260663 18357 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a065545ac85450089dd1b7ed59e03fe: No bootstrap required, opened a new log
I20260811 06:33:00.261632 18599 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 892ce2df978547d0b50aee4db0626860: No bootstrap required, opened a new log
I20260811 06:33:00.268146 18357 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3a065545ac85450089dd1b7ed59e03fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a065545ac85450089dd1b7ed59e03fe" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 34603 } } peers { permanent_uuid: "205eaac6700e484aa2429c74bf9d5f97" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 42215 } } peers { permanent_uuid: "892ce2df978547d0b50aee4db0626860" member_type: VOTER last_known_addr { host: "127.28.239.252" port: 38959 } }
I20260811 06:33:00.268514 18357 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3a065545ac85450089dd1b7ed59e03fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:33:00.268638 18357 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3a065545ac85450089dd1b7ed59e03fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a065545ac85450089dd1b7ed59e03fe, State: Initialized, Role: FOLLOWER
I20260811 06:33:00.269564 18357 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3a065545ac85450089dd1b7ed59e03fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a065545ac85450089dd1b7ed59e03fe" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 34603 } } peers { permanent_uuid: "205eaac6700e484aa2429c74bf9d5f97" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 42215 } } peers { permanent_uuid: "892ce2df978547d0b50aee4db0626860" member_type: VOTER last_known_addr { host: "127.28.239.252" port: 38959 } }
I20260811 06:33:00.270617 18621 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a065545ac85450089dd1b7ed59e03fe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a065545ac85450089dd1b7ed59e03fe" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 34603 } } peers { permanent_uuid: "205eaac6700e484aa2429c74bf9d5f97" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 42215 } } peers { permanent_uuid: "892ce2df978547d0b50aee4db0626860" member_type: VOTER last_known_addr { host: "127.28.239.252" port: 38959 } } }
I20260811 06:33:00.271121 18621 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a065545ac85450089dd1b7ed59e03fe [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:33:00.272173 18357 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3a065545ac85450089dd1b7ed59e03fe [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:33:00.275517 18599 raft_consensus.cc:348] T 00000000000000000000000000000000 P 892ce2df978547d0b50aee4db0626860 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a065545ac85450089dd1b7ed59e03fe" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 34603 } } peers { permanent_uuid: "205eaac6700e484aa2429c74bf9d5f97" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 42215 } } peers { permanent_uuid: "892ce2df978547d0b50aee4db0626860" member_type: VOTER last_known_addr { host: "127.28.239.252" port: 38959 } }
I20260811 06:33:00.276136 18599 raft_consensus.cc:374] T 00000000000000000000000000000000 P 892ce2df978547d0b50aee4db0626860 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:33:00.276268 18599 raft_consensus.cc:729] T 00000000000000000000000000000000 P 892ce2df978547d0b50aee4db0626860 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 892ce2df978547d0b50aee4db0626860, State: Initialized, Role: FOLLOWER
I20260811 06:33:00.277226 18599 consensus_queue.cc:260] T 00000000000000000000000000000000 P 892ce2df978547d0b50aee4db0626860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a065545ac85450089dd1b7ed59e03fe" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 34603 } } peers { permanent_uuid: "205eaac6700e484aa2429c74bf9d5f97" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 42215 } } peers { permanent_uuid: "892ce2df978547d0b50aee4db0626860" member_type: VOTER last_known_addr { host: "127.28.239.252" port: 38959 } }
I20260811 06:33:00.284443 18626 sys_catalog.cc:455] T 00000000000000000000000000000000 P 892ce2df978547d0b50aee4db0626860 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a065545ac85450089dd1b7ed59e03fe" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 34603 } } peers { permanent_uuid: "205eaac6700e484aa2429c74bf9d5f97" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 42215 } } peers { permanent_uuid: "892ce2df978547d0b50aee4db0626860" member_type: VOTER last_known_addr { host: "127.28.239.252" port: 38959 } } }
I20260811 06:33:00.284947 18626 sys_catalog.cc:458] T 00000000000000000000000000000000 P 892ce2df978547d0b50aee4db0626860 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:33:00.287520 18599 sys_catalog.cc:564] T 00000000000000000000000000000000 P 892ce2df978547d0b50aee4db0626860 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:33:00.289198 18626 raft_consensus.cc:482] T 00000000000000000000000000000000 P 892ce2df978547d0b50aee4db0626860 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:33:00.289490 18626 raft_consensus.cc:504] T 00000000000000000000000000000000 P 892ce2df978547d0b50aee4db0626860 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a065545ac85450089dd1b7ed59e03fe" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 34603 } } peers { permanent_uuid: "205eaac6700e484aa2429c74bf9d5f97" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 42215 } } peers { permanent_uuid: "892ce2df978547d0b50aee4db0626860" member_type: VOTER last_known_addr { host: "127.28.239.252" port: 38959 } }
I20260811 06:33:00.292527 18318 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "892ce2df978547d0b50aee4db0626860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a065545ac85450089dd1b7ed59e03fe" is_pre_election: true
I20260811 06:33:00.293265 18318 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3a065545ac85450089dd1b7ed59e03fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 892ce2df978547d0b50aee4db0626860 in term 0.
I20260811 06:33:00.294061 18523 leader_election.cc:302] T 00000000000000000000000000000000 P 892ce2df978547d0b50aee4db0626860 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a065545ac85450089dd1b7ed59e03fe, 892ce2df978547d0b50aee4db0626860; no voters:
I20260811 06:33:00.297639 18606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:00.271373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57338 (local address 127.28.239.254:34603)
0811 06:33:00.271569 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 06:33:00.271587 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 06:33:00.272414 (+ 827us) server_negotiation.cc:407] Connection header received
0811 06:33:00.272723 (+ 309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:33:00.272728 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:33:00.272814 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:33:00.272896 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:33:00.275972 (+ 3076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:00.277562 (+ 1590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:00.280591 (+ 3029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:00.281587 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:00.281726 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:00.287059 (+ 5333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:33:00.287070 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:33:00.287076 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:33:00.287136 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:33:00.292408 (+ 5272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:33:00.294043 (+ 1635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:33:00.294054 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:33:00.294060 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:33:00.294394 (+ 334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:33:00.295015 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:33:00.295023 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:33:00.295029 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:33:00.295593 (+ 564us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:33:00.295932 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0811 06:33:00.296751 (+ 819us) server_negotiation.cc:299] Negotiation successful
0811 06:33:00.297083 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 06:33:00.305209 18626 leader_election.cc:288] T 00000000000000000000000000000000 P 892ce2df978547d0b50aee4db0626860 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a065545ac85450089dd1b7ed59e03fe (127.28.239.254:34603), 205eaac6700e484aa2429c74bf9d5f97 (127.28.239.253:42215)
I20260811 06:33:00.306886 18626 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 892ce2df978547d0b50aee4db0626860 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:33:00.307118 18626 raft_consensus.cc:482] T 00000000000000000000000000000000 P 892ce2df978547d0b50aee4db0626860 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:33:00.307258 18626 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 892ce2df978547d0b50aee4db0626860 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:33:00.314630 18626 raft_consensus.cc:504] T 00000000000000000000000000000000 P 892ce2df978547d0b50aee4db0626860 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a065545ac85450089dd1b7ed59e03fe" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 34603 } } peers { permanent_uuid: "205eaac6700e484aa2429c74bf9d5f97" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 42215 } } peers { permanent_uuid: "892ce2df978547d0b50aee4db0626860" member_type: VOTER last_known_addr { host: "127.28.239.252" port: 38959 } }
I20260811 06:33:00.316061 18318 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "892ce2df978547d0b50aee4db0626860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a065545ac85450089dd1b7ed59e03fe"
I20260811 06:33:00.316349 18318 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3a065545ac85450089dd1b7ed59e03fe [term 0 FOLLOWER]: Advancing to term 1
W20260811 06:33:00.318328 18282 tablet.cc:2369] T 00000000000000000000000000000000 P 3a065545ac85450089dd1b7ed59e03fe: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:33:00.319813 18488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:00.272840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34266 (local address 127.28.239.253:42215)
0811 06:33:00.273017 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 06:33:00.273025 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:33:00.293716 (+ 20691us) server_negotiation.cc:407] Connection header received
0811 06:33:00.294310 (+ 594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:33:00.294317 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:33:00.294408 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:33:00.294493 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:33:00.297737 (+ 3244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:00.299262 (+ 1525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:00.304188 (+ 4926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:00.305064 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:00.305185 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:00.310473 (+ 5288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:33:00.310482 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:33:00.310486 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:33:00.310537 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:33:00.314523 (+ 3986us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:33:00.316054 (+ 1531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:33:00.316059 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:33:00.316062 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:33:00.316263 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:33:00.317104 (+ 841us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:33:00.317111 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:33:00.317116 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:33:00.317607 (+ 491us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:33:00.317867 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0811 06:33:00.318861 (+ 994us) server_negotiation.cc:299] Negotiation successful
0811 06:33:00.319264 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 06:33:00.320137 18318 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3a065545ac85450089dd1b7ed59e03fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 892ce2df978547d0b50aee4db0626860 in term 1.
I20260811 06:33:00.320740 18523 leader_election.cc:302] T 00000000000000000000000000000000 P 892ce2df978547d0b50aee4db0626860 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a065545ac85450089dd1b7ed59e03fe, 892ce2df978547d0b50aee4db0626860; no voters:
I20260811 06:33:00.321097 18632 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 892ce2df978547d0b50aee4db0626860 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:33:00.322407 18607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:00.278220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53226 (local address 127.28.239.252:38959)
0811 06:33:00.278414 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 06:33:00.278420 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:33:00.297451 (+ 19031us) server_negotiation.cc:407] Connection header received
0811 06:33:00.297686 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:33:00.297694 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:33:00.297796 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:33:00.297886 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:33:00.300692 (+ 2806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:00.302221 (+ 1529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:00.304878 (+ 2657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:00.305853 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:00.305985 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:00.311035 (+ 5050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:33:00.311046 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:33:00.311054 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:33:00.311126 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:33:00.316477 (+ 5351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:33:00.317691 (+ 1214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:33:00.317700 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:33:00.317706 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:33:00.317903 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:33:00.319972 (+ 2069us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:33:00.319980 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:33:00.319986 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:33:00.320442 (+ 456us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:33:00.320731 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0811 06:33:00.321548 (+ 817us) server_negotiation.cc:299] Negotiation successful
0811 06:33:00.321904 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 06:33:00.327070 18632 raft_consensus.cc:686] T 00000000000000000000000000000000 P 892ce2df978547d0b50aee4db0626860 [term 1 LEADER]: Becoming Leader. State: Replica: 892ce2df978547d0b50aee4db0626860, State: Running, Role: LEADER
I20260811 06:33:00.327802 18632 consensus_queue.cc:237] T 00000000000000000000000000000000 P 892ce2df978547d0b50aee4db0626860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a065545ac85450089dd1b7ed59e03fe" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 34603 } } peers { permanent_uuid: "205eaac6700e484aa2429c74bf9d5f97" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 42215 } } peers { permanent_uuid: "892ce2df978547d0b50aee4db0626860" member_type: VOTER last_known_addr { host: "127.28.239.252" port: 38959 } }
I20260811 06:33:00.329491 18626 leader_election.cc:288] T 00000000000000000000000000000000 P 892ce2df978547d0b50aee4db0626860 [CANDIDATE]: Term 1 election: Requested vote from peers 3a065545ac85450089dd1b7ed59e03fe (127.28.239.254:34603), 205eaac6700e484aa2429c74bf9d5f97 (127.28.239.253:42215)
I20260811 06:33:00.335281 18636 sys_catalog.cc:455] T 00000000000000000000000000000000 P 892ce2df978547d0b50aee4db0626860 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 892ce2df978547d0b50aee4db0626860. Latest consensus state: current_term: 1 leader_uuid: "892ce2df978547d0b50aee4db0626860" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a065545ac85450089dd1b7ed59e03fe" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 34603 } } peers { permanent_uuid: "205eaac6700e484aa2429c74bf9d5f97" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 42215 } } peers { permanent_uuid: "892ce2df978547d0b50aee4db0626860" member_type: VOTER last_known_addr { host: "127.28.239.252" port: 38959 } } }
I20260811 06:33:00.335805 18636 sys_catalog.cc:458] T 00000000000000000000000000000000 P 892ce2df978547d0b50aee4db0626860 [sys.catalog]: This master's current role is: LEADER
I20260811 06:33:00.341260 18647 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:33:00.348764 18477 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.239.252" port: 38959 } attempt: 5
I20260811 06:33:00.349485 18647 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 11 06:33:00 dist-test-slave-gkw7 krb5kdc[18223](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429979, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.253@KRBTEST.COM for kudu/127.28.239.252@KRBTEST.COM
I20260811 06:33:00.377388 18318 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3a065545ac85450089dd1b7ed59e03fe [term 1 FOLLOWER]: Refusing update from remote peer 892ce2df978547d0b50aee4db0626860: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:33:00.379421 18636 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 892ce2df978547d0b50aee4db0626860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a065545ac85450089dd1b7ed59e03fe" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 34603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260811 06:33:00.379391 18493 master_runner.cc:186] Error getting master registration for 127.28.239.254:34603: OK, instance_id { permanent_uuid: "3a065545ac85450089dd1b7ed59e03fe" instance_seqno: 1786429979080654 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 06:33:00.398111 18493 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 06:33:00.404618 18485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:00.349513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.252:38959 (local address 127.0.0.1:53236)
0811 06:33:00.350006 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:00.350031 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 06:33:00.350163 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:33:00.351293 (+ 1130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:33:00.351299 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:33:00.351966 (+ 667us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:33:00.352250 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:33:00.352266 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:33:00.355346 (+ 3080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:33:00.355356 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:33:00.357993 (+ 2637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:33:00.358044 (+ 51us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:33:00.360854 (+ 2810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:33:00.360863 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:33:00.361566 (+ 703us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:00.362974 (+ 1408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:33:00.363041 (+ 67us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:33:00.369117 (+ 6076us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:33:00.386621 (+ 17504us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:33:00.386629 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:33:00.386635 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:33:00.387233 (+ 598us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:33:00.396795 (+ 9562us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:33:00.396802 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:33:00.396809 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:33:00.397115 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:33:00.403453 (+ 6338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:33:00.403460 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:33:00.403673 (+ 213us) client_negotiation.cc:769] Sending connection context
0811 06:33:00.403842 (+ 169us) client_negotiation.cc:240] Negotiation successful
0811 06:33:00.404207 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332}
W20260811 06:33:00.405184 18525 tablet.cc:2369] T 00000000000000000000000000000000 P 892ce2df978547d0b50aee4db0626860: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:33:00.405993 18607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:00.349773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53236 (local address 127.28.239.252:38959)
0811 06:33:00.350442 (+ 669us) server_negotiation.cc:206] Beginning negotiation
0811 06:33:00.350451 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:33:00.350475 (+ 24us) server_negotiation.cc:407] Connection header received
0811 06:33:00.350540 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:33:00.350548 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:33:00.350642 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:33:00.350754 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:33:00.352955 (+ 2201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:00.355050 (+ 2095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:00.358395 (+ 3345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:00.360180 (+ 1785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:00.360349 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:00.369520 (+ 9171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:33:00.369578 (+ 58us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:33:00.369583 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:33:00.369664 (+ 81us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:33:00.380611 (+ 10947us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:33:00.395840 (+ 15229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:33:00.395850 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:33:00.395855 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:33:00.396566 (+ 711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:33:00.397299 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:33:00.397306 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:33:00.397346 (+ 40us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:33:00.403276 (+ 5930us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:33:00.403504 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0811 06:33:00.404303 (+ 799us) server_negotiation.cc:299] Negotiation successful
0811 06:33:00.405031 (+ 728us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":515}
I20260811 06:33:00.407464 18621 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a065545ac85450089dd1b7ed59e03fe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 892ce2df978547d0b50aee4db0626860. Latest consensus state: current_term: 1 leader_uuid: "892ce2df978547d0b50aee4db0626860" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a065545ac85450089dd1b7ed59e03fe" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 34603 } } peers { permanent_uuid: "205eaac6700e484aa2429c74bf9d5f97" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 42215 } } peers { permanent_uuid: "892ce2df978547d0b50aee4db0626860" member_type: VOTER last_known_addr { host: "127.28.239.252" port: 38959 } } }
I20260811 06:33:00.407832 18621 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a065545ac85450089dd1b7ed59e03fe [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:33:00.412125 18477 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 205eaac6700e484aa2429c74bf9d5f97: Bootstrap starting.
I20260811 06:33:00.429589 18477 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 205eaac6700e484aa2429c74bf9d5f97: Neither blocks nor log segments found. Creating new log.
I20260811 06:33:00.432283 18662 mvcc.cc:205] Tried to move back new op lower bound from 7317217199599173632 to 7317217199430746112. Current Snapshot: MvccSnapshot[applied={T|T < 7317217199599173632}]
I20260811 06:33:00.434608 18663 mvcc.cc:205] Tried to move back new op lower bound from 7317217199599173632 to 7317217199430746112. Current Snapshot: MvccSnapshot[applied={T|T < 7317217199599173632}]
I20260811 06:33:00.436461 18477 log.cc:824] T 00000000000000000000000000000000 P 205eaac6700e484aa2429c74bf9d5f97: Log is configured to *not* fsync() on all Append() calls
I20260811 06:33:00.439426 18626 sys_catalog.cc:455] T 00000000000000000000000000000000 P 892ce2df978547d0b50aee4db0626860 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "892ce2df978547d0b50aee4db0626860" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a065545ac85450089dd1b7ed59e03fe" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 34603 } } peers { permanent_uuid: "205eaac6700e484aa2429c74bf9d5f97" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 42215 } } peers { permanent_uuid: "892ce2df978547d0b50aee4db0626860" member_type: VOTER last_known_addr { host: "127.28.239.252" port: 38959 } } }
I20260811 06:33:00.439896 18626 sys_catalog.cc:458] T 00000000000000000000000000000000 P 892ce2df978547d0b50aee4db0626860 [sys.catalog]: This master's current role is: LEADER
I20260811 06:33:00.445396 18647 catalog_manager.cc:1347] Generated new cluster ID: d6196a619c734636b0e4e65da2069bab
I20260811 06:33:00.445514 18647 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:33:00.449079 18621 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a065545ac85450089dd1b7ed59e03fe [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "892ce2df978547d0b50aee4db0626860" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a065545ac85450089dd1b7ed59e03fe" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 34603 } } peers { permanent_uuid: "205eaac6700e484aa2429c74bf9d5f97" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 42215 } } peers { permanent_uuid: "892ce2df978547d0b50aee4db0626860" member_type: VOTER last_known_addr { host: "127.28.239.252" port: 38959 } } }
I20260811 06:33:00.449443 18621 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a065545ac85450089dd1b7ed59e03fe [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:33:00.459237 18477 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 205eaac6700e484aa2429c74bf9d5f97: No bootstrap required, opened a new log
I20260811 06:33:00.452538 18244 master_runner.cc:186] Error getting master registration for 127.28.239.253:42215: OK, instance_id { permanent_uuid: "205eaac6700e484aa2429c74bf9d5f97" instance_seqno: 1786429979616963 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 06:33:00.465044 18244 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 06:33:00.475281 18668 catalog_manager.cc:1259] Loaded cluster ID: d6196a619c734636b0e4e65da2069bab
I20260811 06:33:00.475432 18668 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3a065545ac85450089dd1b7ed59e03fe: loading cluster ID for follower catalog manager: success
I20260811 06:33:00.475173 18477 raft_consensus.cc:348] T 00000000000000000000000000000000 P 205eaac6700e484aa2429c74bf9d5f97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a065545ac85450089dd1b7ed59e03fe" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 34603 } } peers { permanent_uuid: "205eaac6700e484aa2429c74bf9d5f97" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 42215 } } peers { permanent_uuid: "892ce2df978547d0b50aee4db0626860" member_type: VOTER last_known_addr { host: "127.28.239.252" port: 38959 } }
I20260811 06:33:00.475687 18477 raft_consensus.cc:374] T 00000000000000000000000000000000 P 205eaac6700e484aa2429c74bf9d5f97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:33:00.475844 18477 raft_consensus.cc:729] T 00000000000000000000000000000000 P 205eaac6700e484aa2429c74bf9d5f97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 205eaac6700e484aa2429c74bf9d5f97, State: Initialized, Role: FOLLOWER
I20260811 06:33:00.476624 18477 consensus_queue.cc:260] T 00000000000000000000000000000000 P 205eaac6700e484aa2429c74bf9d5f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a065545ac85450089dd1b7ed59e03fe" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 34603 } } peers { permanent_uuid: "205eaac6700e484aa2429c74bf9d5f97" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 42215 } } peers { permanent_uuid: "892ce2df978547d0b50aee4db0626860" member_type: VOTER last_known_addr { host: "127.28.239.252" port: 38959 } }
W20260811 06:33:00.477675 18668 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 3a065545ac85450089dd1b7ed59e03fe: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260811 06:33:00.477890 18668 catalog_manager.cc:873] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
I20260811 06:33:00.479665 18675 sys_catalog.cc:455] T 00000000000000000000000000000000 P 205eaac6700e484aa2429c74bf9d5f97 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a065545ac85450089dd1b7ed59e03fe" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 34603 } } peers { permanent_uuid: "205eaac6700e484aa2429c74bf9d5f97" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 42215 } } peers { permanent_uuid: "892ce2df978547d0b50aee4db0626860" member_type: VOTER last_known_addr { host: "127.28.239.252" port: 38959 } } }
I20260811 06:33:00.480175 18675 sys_catalog.cc:458] T 00000000000000000000000000000000 P 205eaac6700e484aa2429c74bf9d5f97 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:33:00.479991 18647 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:33:00.481159 18433 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "892ce2df978547d0b50aee4db0626860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "205eaac6700e484aa2429c74bf9d5f97"
I20260811 06:33:00.481670 18433 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 205eaac6700e484aa2429c74bf9d5f97 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:33:00.483209 18647 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:33:00.485204 18433 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 205eaac6700e484aa2429c74bf9d5f97 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 892ce2df978547d0b50aee4db0626860 in term 1.
I20260811 06:33:00.486783 18432 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 205eaac6700e484aa2429c74bf9d5f97 [term 1 FOLLOWER]: Refusing update from remote peer 892ce2df978547d0b50aee4db0626860: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:33:00.490988 18626 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 892ce2df978547d0b50aee4db0626860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "205eaac6700e484aa2429c74bf9d5f97" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 42215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:33:00.493618 18477 sys_catalog.cc:564] T 00000000000000000000000000000000 P 205eaac6700e484aa2429c74bf9d5f97 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:33:00.496943 18434 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "892ce2df978547d0b50aee4db0626860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "205eaac6700e484aa2429c74bf9d5f97" is_pre_election: true
I20260811 06:33:00.528412 18675 sys_catalog.cc:455] T 00000000000000000000000000000000 P 205eaac6700e484aa2429c74bf9d5f97 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 892ce2df978547d0b50aee4db0626860. Latest consensus state: current_term: 1 leader_uuid: "892ce2df978547d0b50aee4db0626860" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a065545ac85450089dd1b7ed59e03fe" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 34603 } } peers { permanent_uuid: "205eaac6700e484aa2429c74bf9d5f97" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 42215 } } peers { permanent_uuid: "892ce2df978547d0b50aee4db0626860" member_type: VOTER last_known_addr { host: "127.28.239.252" port: 38959 } } }
I20260811 06:33:00.528800 18675 sys_catalog.cc:458] T 00000000000000000000000000000000 P 205eaac6700e484aa2429c74bf9d5f97 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:33:00.529292 18682 sys_catalog.cc:455] T 00000000000000000000000000000000 P 205eaac6700e484aa2429c74bf9d5f97 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "892ce2df978547d0b50aee4db0626860" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a065545ac85450089dd1b7ed59e03fe" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 34603 } } peers { permanent_uuid: "205eaac6700e484aa2429c74bf9d5f97" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 42215 } } peers { permanent_uuid: "892ce2df978547d0b50aee4db0626860" member_type: VOTER last_known_addr { host: "127.28.239.252" port: 38959 } } }
I20260811 06:33:00.529821 18682 sys_catalog.cc:458] T 00000000000000000000000000000000 P 205eaac6700e484aa2429c74bf9d5f97 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:33:00.531148 18647 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 892ce2df978547d0b50aee4db0626860: Generated new TSK 0
I20260811 06:33:00.532002 18626 sys_catalog.cc:455] T 00000000000000000000000000000000 P 892ce2df978547d0b50aee4db0626860 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "892ce2df978547d0b50aee4db0626860" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a065545ac85450089dd1b7ed59e03fe" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 34603 } } peers { permanent_uuid: "205eaac6700e484aa2429c74bf9d5f97" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 42215 } } peers { permanent_uuid: "892ce2df978547d0b50aee4db0626860" member_type: VOTER last_known_addr { host: "127.28.239.252" port: 38959 } } }
I20260811 06:33:00.532305 18626 sys_catalog.cc:458] T 00000000000000000000000000000000 P 892ce2df978547d0b50aee4db0626860 [sys.catalog]: This master's current role is: LEADER
I20260811 06:33:00.532514 18647 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:33:00.614188 18710 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 06:33:01.481932 18668 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3a065545ac85450089dd1b7ed59e03fe: acquiring CA information for follower catalog manager: success
I20260811 06:33:01.484171 18668 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3a065545ac85450089dd1b7ed59e03fe: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 06:33:01.618914 18710 catalog_manager.cc:1259] Loaded cluster ID: d6196a619c734636b0e4e65da2069bab
I20260811 06:33:01.619041 18710 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 205eaac6700e484aa2429c74bf9d5f97: loading cluster ID for follower catalog manager: success
I20260811 06:33:01.621815 18710 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 205eaac6700e484aa2429c74bf9d5f97: acquiring CA information for follower catalog manager: success
I20260811 06:33:01.623939 18710 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 205eaac6700e484aa2429c74bf9d5f97: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 06:33:01.675855 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 18244
I20260811 06:33:01.678004 18349 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:33:01.792906 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 18244
I20260811 06:33:01.831985 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 18369
I20260811 06:33:01.833357 18466 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:33:01.935335 18717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:01.934374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:34603 (local address 127.0.0.1:57352)
0811 06:33:01.934985 (+ 611us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:01.935158 (+ 173us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:34603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":396,"threads_started":1}
W20260811 06:33:01.935719 18523 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.28.239.254:34603: connect: Connection refused (error 111)
I20260811 06:33:01.937302 18717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:01.936924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:34603 (local address 127.0.0.1:57360)
0811 06:33:01.937083 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:01.937171 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:34603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260811 06:33:01.937666 18523 consensus_peers.cc:487] T 00000000000000000000000000000000 P 892ce2df978547d0b50aee4db0626860 -> Peer 3a065545ac85450089dd1b7ed59e03fe (127.28.239.254:34603): Couldn't send request to peer 3a065545ac85450089dd1b7ed59e03fe. Status: Network error: Client connection negotiation failed: client connection to 127.28.239.254:34603: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 06:33:01.943598 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 18369
I20260811 06:33:01.971505 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 18493
I20260811 06:33:01.972749 18591 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:33:02.087886 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 18493
2026-08-11T06:33:02Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (5201 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786429755927508-29631-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:33:02 dist-test-slave-gkw7 krb5kdc[18722](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:33:02 dist-test-slave-gkw7 krb5kdc[18722](info): set up 2 sockets
Aug 11 06:33:02 dist-test-slave-gkw7 krb5kdc[18722](info): commencing operation
krb5kdc: starting...
W20260811 06:33:03.521203 29631 mini_kdc.cc:121] Time spent starting KDC: real 1.377s user 0.002s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786429755927508-29631-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786429755927508-29631-0/krb5kdc/test-user.keytab.
Aug 11 06:33:03 dist-test-slave-gkw7 krb5kdc[18722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429983, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:33:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:33:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.239.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.239.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.239.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab.
Entry for principal kudu/127.28.239.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab.
I20260811 06:33:03.835654 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786429755927508-29631-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786429755927508-29631-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786429755927508-29631-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:45317
--webserver_interface=127.28.239.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab
--principal=kudu/127.28.239.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:45831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.239.254:45317
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:33:04.197494 18739 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:33:04.197893 18739 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:33:04.197993 18739 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:33:04.205710 18739 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:33:04.205842 18739 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:33:04.205891 18739 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:33:04.205945 18739 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:33:04.205991 18739 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:33:04.216559 18739 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:45831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786429755927508-29631-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786429755927508-29631-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.239.254:45317
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab
--principal=kudu/127.28.239.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:45317
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.239.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786429755927508-29631-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:33:04.218068 18739 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:33:04.219404 18739 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:33:04.232370 18755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:33:04.232285 18751 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:33:04.232335 18752 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:33:04.233132 18739 server_base.cc:1034] running on GCE node
I20260811 06:33:04.234027 18739 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:33:04.235503 18739 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:33:04.236744 18739 hybrid_clock.cc:648] HybridClock initialized: now 1786429984236659 us; error 57 us; skew 500 ppm
I20260811 06:33:04.237134 18739 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:33:04 dist-test-slave-gkw7 krb5kdc[18722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429984, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:33:04.244906 18739 init.cc:377] Logged in from keytab as kudu/127.28.239.254@KRBTEST.COM (short username kudu)
I20260811 06:33:04.246757 18739 webserver.cc:460] Webserver started at http://127.28.239.254:43279/ using document root <none> and password file <none>
I20260811 06:33:04.247525 18739 fs_manager.cc:359] Metadata directory not provided
I20260811 06:33:04.247692 18739 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:33:04.248040 18739 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:33:04.251402 18739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786429755927508-29631-0/minicluster-data/master-0/data/instance:
uuid: "a9c48ea649d44a9c8034d0a8c4217cb0"
format_stamp: "Formatted at 2026-08-11 06:33:04 on dist-test-slave-gkw7"
server_key: "6489b59ac92bc86fb10cb9dce635e07f"
server_key_iv: "e60f8a82c31c552d50d2be341261186e"
server_key_version: "encryptionkey@0"
I20260811 06:33:04.252306 18739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786429755927508-29631-0/minicluster-data/master-0/wal/instance:
uuid: "a9c48ea649d44a9c8034d0a8c4217cb0"
format_stamp: "Formatted at 2026-08-11 06:33:04 on dist-test-slave-gkw7"
server_key: "6489b59ac92bc86fb10cb9dce635e07f"
server_key_iv: "e60f8a82c31c552d50d2be341261186e"
server_key_version: "encryptionkey@0"
I20260811 06:33:04.258234 18739 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 06:33:04.262423 18766 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:33:04.263995 18739 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 06:33:04.264245 18739 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786429755927508-29631-0/minicluster-data/master-0/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786429755927508-29631-0/minicluster-data/master-0/wal
uuid: "a9c48ea649d44a9c8034d0a8c4217cb0"
format_stamp: "Formatted at 2026-08-11 06:33:04 on dist-test-slave-gkw7"
server_key: "6489b59ac92bc86fb10cb9dce635e07f"
server_key_iv: "e60f8a82c31c552d50d2be341261186e"
server_key_version: "encryptionkey@0"
I20260811 06:33:04.264477 18739 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786429755927508-29631-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786429755927508-29631-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786429755927508-29631-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:33:04.305124 18739 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:33:04.310129 18739 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:33:04.310472 18739 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:33:04.331419 18739 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.254:45317
I20260811 06:33:04.331444 18849 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.254:45317 every 8 connection(s)
I20260811 06:33:04.332917 18739 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
I20260811 06:33:04.336860 18850 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:33:04.338325 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 18739
I20260811 06:33:04.338579 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786429755927508-29631-0/minicluster-data/master-0/wal/instance
I20260811 06:33:04.339447 29631 external_mini_cluster.cc:1428] Setting key 4ea39fb0e301e2459b2693f6cc1fca55
I20260811 06:33:04.352164 18850 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9c48ea649d44a9c8034d0a8c4217cb0: Bootstrap starting.
Aug 11 06:33:04 dist-test-slave-gkw7 krb5kdc[18722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429983, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:33:04.356295 18850 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a9c48ea649d44a9c8034d0a8c4217cb0: Neither blocks nor log segments found. Creating new log.
I20260811 06:33:04.357515 18850 log.cc:824] T 00000000000000000000000000000000 P a9c48ea649d44a9c8034d0a8c4217cb0: Log is configured to *not* fsync() on all Append() calls
I20260811 06:33:04.360764 18850 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9c48ea649d44a9c8034d0a8c4217cb0: No bootstrap required, opened a new log
I20260811 06:33:04.364586 18854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:04.341863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59528 (local address 127.28.239.254:45317)
0811 06:33:04.342821 (+ 958us) server_negotiation.cc:206] Beginning negotiation
0811 06:33:04.342832 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 06:33:04.342875 (+ 43us) server_negotiation.cc:407] Connection header received
0811 06:33:04.344582 (+ 1707us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:33:04.344608 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:33:04.345013 (+ 405us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:33:04.345343 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:33:04.347391 (+ 2048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:04.348741 (+ 1350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:04.350737 (+ 1996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:04.351466 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:04.351600 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:04.356832 (+ 5232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:33:04.356850 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:33:04.356868 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:33:04.356913 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:33:04.360185 (+ 3272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:33:04.360973 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:33:04.360980 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:33:04.360986 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:33:04.361178 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:33:04.361646 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:33:04.361651 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:33:04.361653 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:33:04.362234 (+ 581us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:33:04.362596 (+ 362us) server_negotiation.cc:1035] Waiting for connection context
0811 06:33:04.363101 (+ 505us) server_negotiation.cc:299] Negotiation successful
0811 06:33:04.363557 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":725,"thread_start_us":448,"threads_started":1}
I20260811 06:33:04.368889 29631 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 06:33:04.370079 18850 raft_consensus.cc:348] T 00000000000000000000000000000000 P a9c48ea649d44a9c8034d0a8c4217cb0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9c48ea649d44a9c8034d0a8c4217cb0" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 45317 } }
I20260811 06:33:04.370481 18850 raft_consensus.cc:374] T 00000000000000000000000000000000 P a9c48ea649d44a9c8034d0a8c4217cb0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:33:04.370599 18850 raft_consensus.cc:729] T 00000000000000000000000000000000 P a9c48ea649d44a9c8034d0a8c4217cb0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9c48ea649d44a9c8034d0a8c4217cb0, State: Initialized, Role: FOLLOWER
I20260811 06:33:04.371155 18850 consensus_queue.cc:260] T 00000000000000000000000000000000 P a9c48ea649d44a9c8034d0a8c4217cb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9c48ea649d44a9c8034d0a8c4217cb0" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 45317 } }
I20260811 06:33:04.371413 18850 raft_consensus.cc:388] T 00000000000000000000000000000000 P a9c48ea649d44a9c8034d0a8c4217cb0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:33:04.371565 18850 raft_consensus.cc:482] T 00000000000000000000000000000000 P a9c48ea649d44a9c8034d0a8c4217cb0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:33:04.371763 18850 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a9c48ea649d44a9c8034d0a8c4217cb0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:33:04.374313 18850 raft_consensus.cc:504] T 00000000000000000000000000000000 P a9c48ea649d44a9c8034d0a8c4217cb0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9c48ea649d44a9c8034d0a8c4217cb0" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 45317 } }
I20260811 06:33:04.374897 18850 leader_election.cc:302] T 00000000000000000000000000000000 P a9c48ea649d44a9c8034d0a8c4217cb0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a9c48ea649d44a9c8034d0a8c4217cb0; no voters:
I20260811 06:33:04.375505 18850 leader_election.cc:288] T 00000000000000000000000000000000 P a9c48ea649d44a9c8034d0a8c4217cb0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:33:04.375658 18858 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a9c48ea649d44a9c8034d0a8c4217cb0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:33:04.376432 18858 raft_consensus.cc:686] T 00000000000000000000000000000000 P a9c48ea649d44a9c8034d0a8c4217cb0 [term 1 LEADER]: Becoming Leader. State: Replica: a9c48ea649d44a9c8034d0a8c4217cb0, State: Running, Role: LEADER
I20260811 06:33:04.376953 18858 consensus_queue.cc:237] T 00000000000000000000000000000000 P a9c48ea649d44a9c8034d0a8c4217cb0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9c48ea649d44a9c8034d0a8c4217cb0" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 45317 } }
I20260811 06:33:04.377800 18850 sys_catalog.cc:564] T 00000000000000000000000000000000 P a9c48ea649d44a9c8034d0a8c4217cb0 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:33:04.379984 18859 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9c48ea649d44a9c8034d0a8c4217cb0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a9c48ea649d44a9c8034d0a8c4217cb0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9c48ea649d44a9c8034d0a8c4217cb0" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 45317 } } }
I20260811 06:33:04.380380 18859 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9c48ea649d44a9c8034d0a8c4217cb0 [sys.catalog]: This master's current role is: LEADER
I20260811 06:33:04.379933 18860 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9c48ea649d44a9c8034d0a8c4217cb0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a9c48ea649d44a9c8034d0a8c4217cb0. Latest consensus state: current_term: 1 leader_uuid: "a9c48ea649d44a9c8034d0a8c4217cb0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9c48ea649d44a9c8034d0a8c4217cb0" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 45317 } } }
I20260811 06:33:04.380954 18860 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9c48ea649d44a9c8034d0a8c4217cb0 [sys.catalog]: This master's current role is: LEADER
I20260811 06:33:04.382635 18865 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:33:04.388295 18865 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:33:04.397424 18865 catalog_manager.cc:1347] Generated new cluster ID: 5f2db12ed2124e92ba274bd65393759b
I20260811 06:33:04.397574 18865 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:33:04.413347 18865 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:33:04.414912 18865 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:33:04.429780 18865 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a9c48ea649d44a9c8034d0a8c4217cb0: Generated new TSK 0
I20260811 06:33:04.430763 18865 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:33:04.482681 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 18739
I20260811 06:33:04.483989 18843 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:33:04.584493 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 18739
2026-08-11T06:33:04Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2488 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:33:04 dist-test-slave-gkw7 krb5kdc[18901](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:33:04 dist-test-slave-gkw7 krb5kdc[18901](info): set up 2 sockets
Aug 11 06:33:04 dist-test-slave-gkw7 krb5kdc[18901](info): commencing operation
krb5kdc: starting...
W20260811 06:33:05.937638 29631 mini_kdc.cc:121] Time spent starting KDC: real 1.306s user 0.000s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/krb5kdc/test-user.keytab.
Aug 11 06:33:06 dist-test-slave-gkw7 krb5kdc[18901](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429986, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:33:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:33:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.239.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.239.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.239.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab.
Entry for principal kudu/127.28.239.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab.
I20260811 06:33:06.247152 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:33729
--webserver_interface=127.28.239.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab
--principal=kudu/127.28.239.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:42797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.239.254:33729
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:33:06.601029 18921 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:33:06.601330 18921 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:33:06.601423 18921 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:33:06.608706 18921 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:33:06.608816 18921 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:33:06.608865 18921 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:33:06.608923 18921 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:33:06.608968 18921 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:33:06.620081 18921 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:42797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.239.254:33729
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab
--principal=kudu/127.28.239.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:33729
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.239.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:33:06.621415 18921 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:33:06.622813 18921 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:33:06.635272 18937 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:33:06.635159 18931 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:33:06.635422 18933 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:33:06.636464 18921 server_base.cc:1034] running on GCE node
I20260811 06:33:06.637239 18921 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:33:06.638729 18921 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:33:06.639927 18921 hybrid_clock.cc:648] HybridClock initialized: now 1786429986639871 us; error 54 us; skew 500 ppm
I20260811 06:33:06.640302 18921 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:33:06 dist-test-slave-gkw7 krb5kdc[18901](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429986, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:33:06.643617 18921 init.cc:377] Logged in from keytab as kudu/127.28.239.254@KRBTEST.COM (short username kudu)
I20260811 06:33:06.645118 18921 webserver.cc:460] Webserver started at http://127.28.239.254:40223/ using document root <none> and password file <none>
I20260811 06:33:06.645882 18921 fs_manager.cc:359] Metadata directory not provided
I20260811 06:33:06.645990 18921 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:33:06.646247 18921 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:33:06.649178 18921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/master-0/data/instance:
uuid: "6fb3717db6bf4269ab7e3d2131cd415a"
format_stamp: "Formatted at 2026-08-11 06:33:06 on dist-test-slave-gkw7"
server_key: "1111ffd9099c338c4d48ad67d7c515af"
server_key_iv: "712e843f0a62e464bc4221f4edddc5ba"
server_key_version: "encryptionkey@0"
I20260811 06:33:06.649986 18921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/master-0/wal/instance:
uuid: "6fb3717db6bf4269ab7e3d2131cd415a"
format_stamp: "Formatted at 2026-08-11 06:33:06 on dist-test-slave-gkw7"
server_key: "1111ffd9099c338c4d48ad67d7c515af"
server_key_iv: "712e843f0a62e464bc4221f4edddc5ba"
server_key_version: "encryptionkey@0"
I20260811 06:33:06.655058 18921 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260811 06:33:06.658780 18950 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:33:06.660204 18921 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 06:33:06.660405 18921 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/master-0/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/master-0/wal
uuid: "6fb3717db6bf4269ab7e3d2131cd415a"
format_stamp: "Formatted at 2026-08-11 06:33:06 on dist-test-slave-gkw7"
server_key: "1111ffd9099c338c4d48ad67d7c515af"
server_key_iv: "712e843f0a62e464bc4221f4edddc5ba"
server_key_version: "encryptionkey@0"
I20260811 06:33:06.660616 18921 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:33:06.699587 18921 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:33:06.704401 18921 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:33:06.704725 18921 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:33:06.724417 18921 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.254:33729
I20260811 06:33:06.724453 19039 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.254:33729 every 8 connection(s)
I20260811 06:33:06.725878 18921 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
I20260811 06:33:06.727452 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 18921
I20260811 06:33:06.727712 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/master-0/wal/instance
I20260811 06:33:06.728570 29631 external_mini_cluster.cc:1428] Setting key 3b3bd5f323b619a66762874dfdef3f85
I20260811 06:33:06.730967 19041 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 06:33:06 dist-test-slave-gkw7 krb5kdc[18901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429986, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:33:06.744903 19041 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6fb3717db6bf4269ab7e3d2131cd415a: Bootstrap starting.
I20260811 06:33:06.748157 19041 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6fb3717db6bf4269ab7e3d2131cd415a: Neither blocks nor log segments found. Creating new log.
I20260811 06:33:06.749257 19041 log.cc:824] T 00000000000000000000000000000000 P 6fb3717db6bf4269ab7e3d2131cd415a: Log is configured to *not* fsync() on all Append() calls
I20260811 06:33:06.752668 19041 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6fb3717db6bf4269ab7e3d2131cd415a: No bootstrap required, opened a new log
I20260811 06:33:06.752851 19045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:06.730786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44216 (local address 127.28.239.254:33729)
0811 06:33:06.731658 (+ 872us) server_negotiation.cc:206] Beginning negotiation
0811 06:33:06.731670 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 06:33:06.731719 (+ 49us) server_negotiation.cc:407] Connection header received
0811 06:33:06.733345 (+ 1626us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:33:06.733367 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:33:06.733826 (+ 459us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:33:06.734180 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:33:06.735999 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:06.737420 (+ 1421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:06.739321 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:06.739999 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:06.740127 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:06.745146 (+ 5019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:33:06.745155 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:33:06.745171 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:33:06.745236 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:33:06.748439 (+ 3203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:33:06.749148 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:33:06.749158 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:33:06.749171 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:33:06.749349 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:33:06.749817 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:33:06.749821 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:33:06.749823 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:33:06.750675 (+ 852us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:33:06.750977 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0811 06:33:06.751450 (+ 473us) server_negotiation.cc:299] Negotiation successful
0811 06:33:06.751824 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":675,"thread_start_us":410,"threads_started":1}
I20260811 06:33:06.760206 19041 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6fb3717db6bf4269ab7e3d2131cd415a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fb3717db6bf4269ab7e3d2131cd415a" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 33729 } }
I20260811 06:33:06.760578 19041 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6fb3717db6bf4269ab7e3d2131cd415a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:33:06.760700 19041 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6fb3717db6bf4269ab7e3d2131cd415a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fb3717db6bf4269ab7e3d2131cd415a, State: Initialized, Role: FOLLOWER
I20260811 06:33:06.761240 19041 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6fb3717db6bf4269ab7e3d2131cd415a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fb3717db6bf4269ab7e3d2131cd415a" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 33729 } }
I20260811 06:33:06.761503 19041 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6fb3717db6bf4269ab7e3d2131cd415a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:33:06.761699 19041 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6fb3717db6bf4269ab7e3d2131cd415a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:33:06.761921 19041 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6fb3717db6bf4269ab7e3d2131cd415a [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:33:06.764128 19041 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6fb3717db6bf4269ab7e3d2131cd415a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fb3717db6bf4269ab7e3d2131cd415a" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 33729 } }
I20260811 06:33:06.764654 19041 leader_election.cc:302] T 00000000000000000000000000000000 P 6fb3717db6bf4269ab7e3d2131cd415a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6fb3717db6bf4269ab7e3d2131cd415a; no voters:
I20260811 06:33:06.765511 19041 leader_election.cc:288] T 00000000000000000000000000000000 P 6fb3717db6bf4269ab7e3d2131cd415a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:33:06.765697 19051 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6fb3717db6bf4269ab7e3d2131cd415a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:33:06.767720 19041 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6fb3717db6bf4269ab7e3d2131cd415a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:33:06.772593 19051 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6fb3717db6bf4269ab7e3d2131cd415a [term 1 LEADER]: Becoming Leader. State: Replica: 6fb3717db6bf4269ab7e3d2131cd415a, State: Running, Role: LEADER
I20260811 06:33:06.773159 19051 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6fb3717db6bf4269ab7e3d2131cd415a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fb3717db6bf4269ab7e3d2131cd415a" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 33729 } }
I20260811 06:33:06.776542 19052 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6fb3717db6bf4269ab7e3d2131cd415a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6fb3717db6bf4269ab7e3d2131cd415a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fb3717db6bf4269ab7e3d2131cd415a" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 33729 } } }
I20260811 06:33:06.776713 19053 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6fb3717db6bf4269ab7e3d2131cd415a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6fb3717db6bf4269ab7e3d2131cd415a. Latest consensus state: current_term: 1 leader_uuid: "6fb3717db6bf4269ab7e3d2131cd415a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fb3717db6bf4269ab7e3d2131cd415a" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 33729 } } }
I20260811 06:33:06.776978 19052 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6fb3717db6bf4269ab7e3d2131cd415a [sys.catalog]: This master's current role is: LEADER
I20260811 06:33:06.777040 19053 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6fb3717db6bf4269ab7e3d2131cd415a [sys.catalog]: This master's current role is: LEADER
W20260811 06:33:06.786149 19067 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6fb3717db6bf4269ab7e3d2131cd415a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 06:33:06.786305 19067 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 06:33:06.786451 19069 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:33:06.787997 19069 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:33:06.796727 19069 catalog_manager.cc:1347] Generated new cluster ID: 610c580947464c11955b0a775094e958
I20260811 06:33:06.796839 19069 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:33:06.822793 19069 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:33:06.824802 19069 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:33:06.843461 19069 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6fb3717db6bf4269ab7e3d2131cd415a: Generated new TSK 0
I20260811 06:33:06.844700 19069 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.239.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.239.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.239.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.193.keytab.
Entry for principal kudu/127.28.239.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.193.keytab.
I20260811 06:33:06.930109 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.193:0
--local_ip_for_outbound_sockets=127.28.239.193
--webserver_interface=127.28.239.193
--webserver_port=0
--tserver_master_addrs=127.28.239.254:33729
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.193.keytab
--principal=kudu/127.28.239.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:42797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:33:07.292225 19078 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:33:07.292542 19078 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:33:07.292637 19078 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:33:07.300210 19078 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:33:07.300347 19078 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:33:07.300469 19078 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.193
I20260811 06:33:07.310680 19078 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:42797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.193.keytab
--principal=kudu/127.28.239.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.239.193
--webserver_port=0
--tserver_master_addrs=127.28.239.254:33729
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.193
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:33:07.312019 19078 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:33:07.313396 19078 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:33:07.327901 19090 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:33:07.327956 19093 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:33:07.328004 19089 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:33:07.331261 19078 server_base.cc:1034] running on GCE node
I20260811 06:33:07.331912 19078 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:33:07.332962 19078 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:33:07.334183 19078 hybrid_clock.cc:648] HybridClock initialized: now 1786429987334136 us; error 47 us; skew 500 ppm
I20260811 06:33:07.334524 19078 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:33:07 dist-test-slave-gkw7 krb5kdc[18901](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429987, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:33:07.338008 19078 init.cc:377] Logged in from keytab as kudu/127.28.239.193@KRBTEST.COM (short username kudu)
I20260811 06:33:07.339828 19078 webserver.cc:460] Webserver started at http://127.28.239.193:46097/ using document root <none> and password file <none>
I20260811 06:33:07.340600 19078 fs_manager.cc:359] Metadata directory not provided
I20260811 06:33:07.340754 19078 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:33:07.341085 19078 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:33:07.344314 19078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-0/data/instance:
uuid: "fc4e99a911a8498fb3fecd21d55491d9"
format_stamp: "Formatted at 2026-08-11 06:33:07 on dist-test-slave-gkw7"
server_key: "87229cf8f4d20f1ce8947f6f5b40d369"
server_key_iv: "1caf3cbbcbf5374eb3fe102526d0da06"
server_key_version: "encryptionkey@0"
I20260811 06:33:07.345157 19078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-0/wal/instance:
uuid: "fc4e99a911a8498fb3fecd21d55491d9"
format_stamp: "Formatted at 2026-08-11 06:33:07 on dist-test-slave-gkw7"
server_key: "87229cf8f4d20f1ce8947f6f5b40d369"
server_key_iv: "1caf3cbbcbf5374eb3fe102526d0da06"
server_key_version: "encryptionkey@0"
I20260811 06:33:07.350816 19078 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260811 06:33:07.354655 19103 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:33:07.356117 19078 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 06:33:07.356326 19078 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-0/wal
uuid: "fc4e99a911a8498fb3fecd21d55491d9"
format_stamp: "Formatted at 2026-08-11 06:33:07 on dist-test-slave-gkw7"
server_key: "87229cf8f4d20f1ce8947f6f5b40d369"
server_key_iv: "1caf3cbbcbf5374eb3fe102526d0da06"
server_key_version: "encryptionkey@0"
I20260811 06:33:07.356559 19078 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:33:07.393314 19078 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:33:07.398242 19078 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:33:07.398554 19078 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:33:07.399634 19078 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:33:07.401069 19078 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:33:07.401194 19078 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:33:07.401348 19078 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:33:07.401425 19078 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:33:07.437238 19078 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.193:37147
I20260811 06:33:07.437292 19274 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.193:37147 every 8 connection(s)
I20260811 06:33:07.438679 19078 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
I20260811 06:33:07.440237 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 19078
I20260811 06:33:07.440472 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-0/wal/instance
I20260811 06:33:07.441224 29631 external_mini_cluster.cc:1428] Setting key ad08b6d2def82536c2be5545716af943
Aug 11 06:33:07 dist-test-slave-gkw7 krb5kdc[18901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429987, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.193@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:33:07.464991 19278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:07.441881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.193:53041 (local address 127.28.239.254:33729)
0811 06:33:07.442441 (+ 560us) server_negotiation.cc:206] Beginning negotiation
0811 06:33:07.442449 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:33:07.443846 (+ 1397us) server_negotiation.cc:407] Connection header received
0811 06:33:07.446918 (+ 3072us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:33:07.446923 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:33:07.446993 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:33:07.447090 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:33:07.449355 (+ 2265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:07.450246 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:07.452863 (+ 2617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:07.453268 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:07.453373 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:07.458074 (+ 4701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:33:07.458079 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:33:07.458082 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:33:07.458116 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:33:07.461436 (+ 3320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:33:07.462305 (+ 869us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:33:07.462312 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:33:07.462316 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:33:07.462460 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:33:07.462955 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:33:07.462961 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:33:07.462965 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:33:07.463275 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:33:07.463453 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0811 06:33:07.464251 (+ 798us) server_negotiation.cc:299] Negotiation successful
0811 06:33:07.464509 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":428,"thread_start_us":284,"threads_started":1}
I20260811 06:33:07.466100 19279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:07.442687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:33729 (local address 127.28.239.193:53041)
0811 06:33:07.443635 (+ 948us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:07.443708 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0811 06:33:07.446587 (+ 2879us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:33:07.447395 (+ 808us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:33:07.447409 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:33:07.448236 (+ 827us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:33:07.449119 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:33:07.449158 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:33:07.450417 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:33:07.450425 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:33:07.452684 (+ 2259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:33:07.452703 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:33:07.453508 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:33:07.453518 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:33:07.453879 (+ 361us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:07.454679 (+ 800us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:33:07.454715 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:33:07.457820 (+ 3105us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:33:07.461632 (+ 3812us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:33:07.461643 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:33:07.461661 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:33:07.462161 (+ 500us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:33:07.462596 (+ 435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:33:07.462602 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:33:07.462606 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:33:07.462827 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:33:07.463427 (+ 600us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:33:07.463436 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:33:07.463819 (+ 383us) client_negotiation.cc:769] Sending connection context
0811 06:33:07.464122 (+ 303us) client_negotiation.cc:240] Negotiation successful
0811 06:33:07.464782 (+ 660us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":647,"thread_start_us":385,"threads_started":1}
I20260811 06:33:07.468010 19275 heartbeater.cc:347] Connected to a master server at 127.28.239.254:33729
I20260811 06:33:07.468428 19275 heartbeater.cc:464] Registering TS with master...
I20260811 06:33:07.469465 19275 heartbeater.cc:511] Master 127.28.239.254:33729 requested a full tablet report, sending...
I20260811 06:33:07.472100 18976 ts_manager.cc:194] Registered new tserver with Master: fc4e99a911a8498fb3fecd21d55491d9 (127.28.239.193:37147)
I20260811 06:33:07.473757 18976 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.239.193@KRBTEST.COM'} at 127.28.239.193:53041
WARNING: no policy specified for kudu/127.28.239.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.239.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.239.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.194.keytab.
Entry for principal kudu/127.28.239.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.194.keytab.
I20260811 06:33:07.511524 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.194:0
--local_ip_for_outbound_sockets=127.28.239.194
--webserver_interface=127.28.239.194
--webserver_port=0
--tserver_master_addrs=127.28.239.254:33729
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.194.keytab
--principal=kudu/127.28.239.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:42797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:33:07.868209 19283 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:33:07.868552 19283 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:33:07.868757 19283 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:33:07.876382 19283 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:33:07.876497 19283 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:33:07.876694 19283 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.194
I20260811 06:33:07.886854 19283 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:42797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.194.keytab
--principal=kudu/127.28.239.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.239.194
--webserver_port=0
--tserver_master_addrs=127.28.239.254:33729
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.194
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:33:07.888278 19283 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:33:07.889725 19283 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:33:07.904590 19288 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:33:07.904659 19291 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:33:07.904774 19295 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:33:07.905380 19283 server_base.cc:1034] running on GCE node
I20260811 06:33:07.906096 19283 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:33:07.907155 19283 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:33:07.908367 19283 hybrid_clock.cc:648] HybridClock initialized: now 1786429987908323 us; error 39 us; skew 500 ppm
I20260811 06:33:07.908663 19283 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:33:07 dist-test-slave-gkw7 krb5kdc[18901](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429987, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:33:07.912235 19283 init.cc:377] Logged in from keytab as kudu/127.28.239.194@KRBTEST.COM (short username kudu)
I20260811 06:33:07.913977 19283 webserver.cc:460] Webserver started at http://127.28.239.194:44943/ using document root <none> and password file <none>
I20260811 06:33:07.914757 19283 fs_manager.cc:359] Metadata directory not provided
I20260811 06:33:07.914896 19283 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:33:07.915236 19283 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:33:07.918411 19283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-1/data/instance:
uuid: "91ec4d92310b4496a89c1eecf66102a3"
format_stamp: "Formatted at 2026-08-11 06:33:07 on dist-test-slave-gkw7"
server_key: "37f6f06d05194a49812e2141c9e9a675"
server_key_iv: "8b0ff52b36eeddcaff3c89b29eb22656"
server_key_version: "encryptionkey@0"
I20260811 06:33:07.919250 19283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-1/wal/instance:
uuid: "91ec4d92310b4496a89c1eecf66102a3"
format_stamp: "Formatted at 2026-08-11 06:33:07 on dist-test-slave-gkw7"
server_key: "37f6f06d05194a49812e2141c9e9a675"
server_key_iv: "8b0ff52b36eeddcaff3c89b29eb22656"
server_key_version: "encryptionkey@0"
I20260811 06:33:07.924952 19283 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260811 06:33:07.929069 19304 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:33:07.930733 19283 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 06:33:07.930951 19283 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-1/wal
uuid: "91ec4d92310b4496a89c1eecf66102a3"
format_stamp: "Formatted at 2026-08-11 06:33:07 on dist-test-slave-gkw7"
server_key: "37f6f06d05194a49812e2141c9e9a675"
server_key_iv: "8b0ff52b36eeddcaff3c89b29eb22656"
server_key_version: "encryptionkey@0"
I20260811 06:33:07.931191 19283 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:33:07.944434 19283 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:33:07.949069 19283 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:33:07.949409 19283 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:33:07.950578 19283 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:33:07.952044 19283 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:33:07.952160 19283 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:33:07.952333 19283 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:33:07.952417 19283 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:33:07.992039 19283 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.194:37153
I20260811 06:33:07.992134 19496 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.194:37153 every 8 connection(s)
I20260811 06:33:07.993479 19283 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
I20260811 06:33:08.002506 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 19283
I20260811 06:33:08.002759 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-1/wal/instance
I20260811 06:33:08.003419 29631 external_mini_cluster.cc:1428] Setting key 1ddcda472f336063ab040b6be3c38c5f
Aug 11 06:33:08 dist-test-slave-gkw7 krb5kdc[18901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429987, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.194@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:33:08.016710 19500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:07.996499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.194:35075 (local address 127.28.239.254:33729)
0811 06:33:07.996968 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0811 06:33:07.996973 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:33:07.997999 (+ 1026us) server_negotiation.cc:407] Connection header received
0811 06:33:08.000174 (+ 2175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:33:08.000180 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:33:08.000249 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:33:08.000354 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:33:08.002137 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:08.003490 (+ 1353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:08.005378 (+ 1888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:08.005948 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:08.006098 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:08.010169 (+ 4071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:33:08.010175 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:33:08.010178 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:33:08.010214 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:33:08.013153 (+ 2939us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:33:08.014095 (+ 942us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:33:08.014101 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:33:08.014103 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:33:08.014236 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:33:08.014733 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:33:08.014744 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:33:08.014747 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:33:08.015062 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:33:08.015239 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0811 06:33:08.016034 (+ 795us) server_negotiation.cc:299] Negotiation successful
0811 06:33:08.016304 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":229,"threads_started":1}
I20260811 06:33:08.017961 19501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:07.996989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:33729 (local address 127.28.239.194:35075)
0811 06:33:07.997823 (+ 834us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:07.997873 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 06:33:07.999835 (+ 1962us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:33:08.000594 (+ 759us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:33:08.000603 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:33:08.001164 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:33:08.001867 (+ 703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:33:08.001888 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:33:08.003650 (+ 1762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:33:08.003654 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:33:08.005212 (+ 1558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:33:08.005231 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:33:08.006218 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:33:08.006227 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:33:08.006476 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:08.007237 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:33:08.007264 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:33:08.009917 (+ 2653us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:33:08.013331 (+ 3414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:33:08.013341 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:33:08.013362 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:33:08.013944 (+ 582us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:33:08.014373 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:33:08.014379 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:33:08.014383 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:33:08.014593 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:33:08.015209 (+ 616us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:33:08.015218 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:33:08.015609 (+ 391us) client_negotiation.cc:769] Sending connection context
0811 06:33:08.015911 (+ 302us) client_negotiation.cc:240] Negotiation successful
0811 06:33:08.016530 (+ 619us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":283,"threads_started":1}
I20260811 06:33:08.019500 19497 heartbeater.cc:347] Connected to a master server at 127.28.239.254:33729
I20260811 06:33:08.019868 19497 heartbeater.cc:464] Registering TS with master...
I20260811 06:33:08.020792 19497 heartbeater.cc:511] Master 127.28.239.254:33729 requested a full tablet report, sending...
I20260811 06:33:08.022624 18976 ts_manager.cc:194] Registered new tserver with Master: 91ec4d92310b4496a89c1eecf66102a3 (127.28.239.194:37153)
I20260811 06:33:08.023561 18976 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.239.194@KRBTEST.COM'} at 127.28.239.194:35075
WARNING: no policy specified for kudu/127.28.239.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.239.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.239.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.195.keytab.
Entry for principal kudu/127.28.239.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.195.keytab.
I20260811 06:33:08.074743 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.195:0
--local_ip_for_outbound_sockets=127.28.239.195
--webserver_interface=127.28.239.195
--webserver_port=0
--tserver_master_addrs=127.28.239.254:33729
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.195.keytab
--principal=kudu/127.28.239.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:42797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:33:08.424108 19505 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:33:08.424408 19505 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:33:08.424503 19505 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:33:08.432047 19505 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:33:08.432169 19505 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:33:08.432345 19505 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.195
I20260811 06:33:08.446357 19505 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:42797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.195.keytab
--principal=kudu/127.28.239.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.239.195
--webserver_port=0
--tserver_master_addrs=127.28.239.254:33729
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.195
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:33:08.448246 19505 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:33:08.450238 19505 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:33:08.466056 19512 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:33:08.466104 19515 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:33:08.466125 19511 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:33:08.471233 19505 server_base.cc:1034] running on GCE node
I20260811 06:33:08.471926 19505 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:33:08.472946 19505 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:33:08.474151 19505 hybrid_clock.cc:648] HybridClock initialized: now 1786429988474100 us; error 44 us; skew 500 ppm
I20260811 06:33:08.474512 19505 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:33:08.476846 19275 heartbeater.cc:503] Master 127.28.239.254:33729 was elected leader, sending a full tablet report...
Aug 11 06:33:08 dist-test-slave-gkw7 krb5kdc[18901](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429988, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:33:08.478304 19505 init.cc:377] Logged in from keytab as kudu/127.28.239.195@KRBTEST.COM (short username kudu)
I20260811 06:33:08.480017 19505 webserver.cc:460] Webserver started at http://127.28.239.195:33353/ using document root <none> and password file <none>
I20260811 06:33:08.480835 19505 fs_manager.cc:359] Metadata directory not provided
I20260811 06:33:08.480991 19505 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:33:08.481341 19505 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:33:08.484735 19505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-2/data/instance:
uuid: "bb42fe5d26884700bb28ac7c1da33596"
format_stamp: "Formatted at 2026-08-11 06:33:08 on dist-test-slave-gkw7"
server_key: "984b476d0ce85290bf9011a0ea33b0b3"
server_key_iv: "e038ec5b7d17787c35cc14ff98ca44b3"
server_key_version: "encryptionkey@0"
I20260811 06:33:08.485741 19505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-2/wal/instance:
uuid: "bb42fe5d26884700bb28ac7c1da33596"
format_stamp: "Formatted at 2026-08-11 06:33:08 on dist-test-slave-gkw7"
server_key: "984b476d0ce85290bf9011a0ea33b0b3"
server_key_iv: "e038ec5b7d17787c35cc14ff98ca44b3"
server_key_version: "encryptionkey@0"
I20260811 06:33:08.491614 19505 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 06:33:08.495721 19527 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:33:08.497161 19505 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 06:33:08.497359 19505 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-2/wal
uuid: "bb42fe5d26884700bb28ac7c1da33596"
format_stamp: "Formatted at 2026-08-11 06:33:08 on dist-test-slave-gkw7"
server_key: "984b476d0ce85290bf9011a0ea33b0b3"
server_key_iv: "e038ec5b7d17787c35cc14ff98ca44b3"
server_key_version: "encryptionkey@0"
I20260811 06:33:08.497692 19505 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:33:08.531582 19505 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:33:08.536484 19505 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:33:08.536826 19505 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:33:08.538053 19505 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:33:08.539666 19505 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:33:08.539749 19505 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:33:08.539945 19505 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:33:08.540030 19505 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:33:08.574784 19505 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.195:36329
I20260811 06:33:08.574820 19704 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.195:36329 every 8 connection(s)
I20260811 06:33:08.576251 19505 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
I20260811 06:33:08.578282 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 19505
I20260811 06:33:08.578485 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-2/wal/instance
I20260811 06:33:08.579030 29631 external_mini_cluster.cc:1428] Setting key b2616d4726c278ba95ba3b8ac0199a99
Aug 11 06:33:08 dist-test-slave-gkw7 krb5kdc[18901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429988, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.195@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:33:08.597961 19709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:08.579589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.195:34137 (local address 127.28.239.254:33729)
0811 06:33:08.580065 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0811 06:33:08.580070 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:33:08.581503 (+ 1433us) server_negotiation.cc:407] Connection header received
0811 06:33:08.583593 (+ 2090us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:33:08.583597 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:33:08.583663 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:33:08.583759 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:33:08.585410 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:08.586249 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:08.587888 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:08.588282 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:08.588373 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:08.591889 (+ 3516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:33:08.591894 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:33:08.591897 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:33:08.591934 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:33:08.595122 (+ 3188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:33:08.595783 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:33:08.595787 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:33:08.595790 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:33:08.595908 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:33:08.596318 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:33:08.596321 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:33:08.596323 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:33:08.596576 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:33:08.596693 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 06:33:08.597425 (+ 732us) server_negotiation.cc:299] Negotiation successful
0811 06:33:08.597667 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":223,"threads_started":1}
I20260811 06:33:08.599053 19710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:08.580322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:33729 (local address 127.28.239.195:34137)
0811 06:33:08.581319 (+ 997us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:08.581372 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 06:33:08.583333 (+ 1961us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:33:08.583988 (+ 655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:33:08.583997 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:33:08.584602 (+ 605us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:33:08.585238 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:33:08.585258 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:33:08.586426 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:33:08.586431 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:33:08.587754 (+ 1323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:33:08.587767 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:33:08.588476 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:33:08.588481 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:33:08.588691 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:08.589307 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:33:08.589329 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:33:08.591667 (+ 2338us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:33:08.595245 (+ 3578us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:33:08.595252 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:33:08.595265 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:33:08.595681 (+ 416us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:33:08.596002 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:33:08.596006 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:33:08.596008 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:33:08.596212 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:33:08.596725 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:33:08.596732 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:33:08.597075 (+ 343us) client_negotiation.cc:769] Sending connection context
0811 06:33:08.597353 (+ 278us) client_negotiation.cc:240] Negotiation successful
0811 06:33:08.597865 (+ 512us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":684,"thread_start_us":441,"threads_started":1}
I20260811 06:33:08.600478 19705 heartbeater.cc:347] Connected to a master server at 127.28.239.254:33729
I20260811 06:33:08.600805 19705 heartbeater.cc:464] Registering TS with master...
I20260811 06:33:08.601712 19705 heartbeater.cc:511] Master 127.28.239.254:33729 requested a full tablet report, sending...
I20260811 06:33:08.603286 18976 ts_manager.cc:194] Registered new tserver with Master: bb42fe5d26884700bb28ac7c1da33596 (127.28.239.195:36329)
I20260811 06:33:08.604121 18976 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.239.195@KRBTEST.COM'} at 127.28.239.195:34137
I20260811 06:33:08.607146 29631 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 06:33:08.624259 19709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:08.611762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44222 (local address 127.28.239.254:33729)
0811 06:33:08.611945 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 06:33:08.611950 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:33:08.612330 (+ 380us) server_negotiation.cc:407] Connection header received
0811 06:33:08.612465 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:33:08.612469 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:33:08.612531 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:33:08.612623 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:33:08.614134 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:08.614875 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:08.616185 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:08.616610 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:08.616716 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:08.619113 (+ 2397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:33:08.619118 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:33:08.619121 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:33:08.619153 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:33:08.621908 (+ 2755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:33:08.622546 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:33:08.622550 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:33:08.622552 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:33:08.622690 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:33:08.623002 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:33:08.623006 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:33:08.623008 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:33:08.623333 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:33:08.623534 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0811 06:33:08.623806 (+ 272us) server_negotiation.cc:299] Negotiation successful
0811 06:33:08.623983 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 06:33:08.628670 18976 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44222:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:33:08.636689 18976 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:33:08.683389 19731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:08.649763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33004 (local address 127.28.239.194:37153)
0811 06:33:08.668444 (+ 18681us) server_negotiation.cc:206] Beginning negotiation
0811 06:33:08.668458 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 06:33:08.669914 (+ 1456us) server_negotiation.cc:407] Connection header received
0811 06:33:08.670060 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:33:08.670072 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:33:08.670463 (+ 391us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:33:08.670691 (+ 228us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:33:08.671985 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:08.675783 (+ 3798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:08.680345 (+ 4562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:08.681985 (+ 1640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:08.682190 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:08.682404 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0811 06:33:08.682835 (+ 431us) server_negotiation.cc:299] Negotiation successful
0811 06:33:08.683017 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18433,"thread_start_us":18710,"threads_started":1}
I20260811 06:33:08.683498 19735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:08.649496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.194:37153 (local address 127.0.0.1:33004)
0811 06:33:08.669335 (+ 19839us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:08.669379 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 06:33:08.669630 (+ 251us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:33:08.671077 (+ 1447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:33:08.671089 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:33:08.671133 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:33:08.671698 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:33:08.671717 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:33:08.677403 (+ 5686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:33:08.677409 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:33:08.679902 (+ 2493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:33:08.679914 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:33:08.682320 (+ 2406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:33:08.682329 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:33:08.682565 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:08.682610 (+ 45us) client_negotiation.cc:769] Sending connection context
0811 06:33:08.682776 (+ 166us) client_negotiation.cc:240] Negotiation successful
0811 06:33:08.682999 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19567,"mutex_wait_us":4,"thread_start_us":19397,"threads_started":1}
I20260811 06:33:08.686347 19737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:08.649633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.193:37147 (local address 127.0.0.1:52290)
0811 06:33:08.676566 (+ 26933us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:08.676592 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 06:33:08.676737 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:33:08.678168 (+ 1431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:33:08.678176 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:33:08.678205 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:33:08.678715 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:33:08.678732 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:33:08.680795 (+ 2063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:33:08.680800 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:33:08.683353 (+ 2553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:33:08.683368 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:33:08.685498 (+ 2130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:33:08.685502 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:33:08.685679 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:08.685703 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 06:33:08.685804 (+ 101us) client_negotiation.cc:240] Negotiation successful
0811 06:33:08.685931 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26745,"mutex_wait_us":22,"thread_start_us":27977,"threads_started":1}
I20260811 06:33:08.686376 19733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:08.649929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52290 (local address 127.28.239.193:37147)
0811 06:33:08.677406 (+ 27477us) server_negotiation.cc:206] Beginning negotiation
0811 06:33:08.677414 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:33:08.677446 (+ 32us) server_negotiation.cc:407] Connection header received
0811 06:33:08.677573 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:33:08.677585 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:33:08.677910 (+ 325us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:33:08.678117 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:33:08.678965 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:08.680630 (+ 1665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:08.683571 (+ 2941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:08.685064 (+ 1493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:08.685255 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:08.685381 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 06:33:08.685863 (+ 482us) server_negotiation.cc:299] Negotiation successful
0811 06:33:08.686036 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27301,"thread_start_us":27119,"threads_started":1}
I20260811 06:33:08.687945 19381 tablet_service.cc:1467] Processing CreateTablet for tablet b7a4572f3919463098951288c395d49d (DEFAULT_TABLE table=test-table [id=28c4fab1e41b4292818028cc1da7a410]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:33:08.688841 19734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:08.650373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.195:36329 (local address 127.0.0.1:54670)
0811 06:33:08.677924 (+ 27551us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:08.677940 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:33:08.678103 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:33:08.678799 (+ 696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:33:08.678805 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:33:08.678831 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:33:08.679269 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:33:08.679282 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:33:08.682492 (+ 3210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:33:08.682497 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:33:08.684994 (+ 2497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:33:08.685011 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:33:08.687975 (+ 2964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:33:08.687982 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:33:08.688158 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:08.688189 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 06:33:08.688308 (+ 119us) client_negotiation.cc:240] Negotiation successful
0811 06:33:08.688448 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27417,"mutex_wait_us":24,"thread_start_us":25758,"threads_started":1}
I20260811 06:33:08.688906 19739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:08.651249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54670 (local address 127.28.239.195:36329)
0811 06:33:08.677072 (+ 25823us) server_negotiation.cc:206] Beginning negotiation
0811 06:33:08.677086 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 06:33:08.678070 (+ 984us) server_negotiation.cc:407] Connection header received
0811 06:33:08.678324 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:33:08.678335 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:33:08.678567 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:33:08.678819 (+ 252us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:33:08.679481 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:08.682312 (+ 2831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:08.685190 (+ 2878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:08.687747 (+ 2557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:08.687958 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:08.688131 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0811 06:33:08.688372 (+ 241us) server_negotiation.cc:299] Negotiation successful
0811 06:33:08.688532 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25565,"thread_start_us":25828,"threads_started":1}
I20260811 06:33:08.691236 19381 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7a4572f3919463098951288c395d49d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:33:08.691145 19177 tablet_service.cc:1467] Processing CreateTablet for tablet b7a4572f3919463098951288c395d49d (DEFAULT_TABLE table=test-table [id=28c4fab1e41b4292818028cc1da7a410]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:33:08.692337 19177 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7a4572f3919463098951288c395d49d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:33:08.692540 19610 tablet_service.cc:1467] Processing CreateTablet for tablet b7a4572f3919463098951288c395d49d (DEFAULT_TABLE table=test-table [id=28c4fab1e41b4292818028cc1da7a410]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:33:08.693966 19610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7a4572f3919463098951288c395d49d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:33:08.703967 19741 tablet_bootstrap.cc:492] T b7a4572f3919463098951288c395d49d P fc4e99a911a8498fb3fecd21d55491d9: Bootstrap starting.
I20260811 06:33:08.706423 19742 tablet_bootstrap.cc:492] T b7a4572f3919463098951288c395d49d P 91ec4d92310b4496a89c1eecf66102a3: Bootstrap starting.
I20260811 06:33:08.707981 19741 tablet_bootstrap.cc:654] T b7a4572f3919463098951288c395d49d P fc4e99a911a8498fb3fecd21d55491d9: Neither blocks nor log segments found. Creating new log.
I20260811 06:33:08.708427 19743 tablet_bootstrap.cc:492] T b7a4572f3919463098951288c395d49d P bb42fe5d26884700bb28ac7c1da33596: Bootstrap starting.
I20260811 06:33:08.710047 19742 tablet_bootstrap.cc:654] T b7a4572f3919463098951288c395d49d P 91ec4d92310b4496a89c1eecf66102a3: Neither blocks nor log segments found. Creating new log.
I20260811 06:33:08.710144 19741 log.cc:824] T b7a4572f3919463098951288c395d49d P fc4e99a911a8498fb3fecd21d55491d9: Log is configured to *not* fsync() on all Append() calls
I20260811 06:33:08.711333 19742 log.cc:824] T b7a4572f3919463098951288c395d49d P 91ec4d92310b4496a89c1eecf66102a3: Log is configured to *not* fsync() on all Append() calls
I20260811 06:33:08.712080 19743 tablet_bootstrap.cc:654] T b7a4572f3919463098951288c395d49d P bb42fe5d26884700bb28ac7c1da33596: Neither blocks nor log segments found. Creating new log.
I20260811 06:33:08.713272 19743 log.cc:824] T b7a4572f3919463098951288c395d49d P bb42fe5d26884700bb28ac7c1da33596: Log is configured to *not* fsync() on all Append() calls
I20260811 06:33:08.713620 19741 tablet_bootstrap.cc:492] T b7a4572f3919463098951288c395d49d P fc4e99a911a8498fb3fecd21d55491d9: No bootstrap required, opened a new log
I20260811 06:33:08.714377 19741 ts_tablet_manager.cc:1397] T b7a4572f3919463098951288c395d49d P fc4e99a911a8498fb3fecd21d55491d9: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.003s
I20260811 06:33:08.714669 19742 tablet_bootstrap.cc:492] T b7a4572f3919463098951288c395d49d P 91ec4d92310b4496a89c1eecf66102a3: No bootstrap required, opened a new log
I20260811 06:33:08.714951 19742 ts_tablet_manager.cc:1397] T b7a4572f3919463098951288c395d49d P 91ec4d92310b4496a89c1eecf66102a3: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260811 06:33:08.719718 19743 tablet_bootstrap.cc:492] T b7a4572f3919463098951288c395d49d P bb42fe5d26884700bb28ac7c1da33596: No bootstrap required, opened a new log
I20260811 06:33:08.719992 19743 ts_tablet_manager.cc:1397] T b7a4572f3919463098951288c395d49d P bb42fe5d26884700bb28ac7c1da33596: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.003s
I20260811 06:33:08.725791 19741 raft_consensus.cc:348] T b7a4572f3919463098951288c395d49d P fc4e99a911a8498fb3fecd21d55491d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ec4d92310b4496a89c1eecf66102a3" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37153 } } peers { permanent_uuid: "fc4e99a911a8498fb3fecd21d55491d9" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 37147 } } peers { permanent_uuid: "bb42fe5d26884700bb28ac7c1da33596" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 36329 } }
I20260811 06:33:08.725785 19742 raft_consensus.cc:348] T b7a4572f3919463098951288c395d49d P 91ec4d92310b4496a89c1eecf66102a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ec4d92310b4496a89c1eecf66102a3" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37153 } } peers { permanent_uuid: "fc4e99a911a8498fb3fecd21d55491d9" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 37147 } } peers { permanent_uuid: "bb42fe5d26884700bb28ac7c1da33596" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 36329 } }
I20260811 06:33:08.726267 19741 raft_consensus.cc:374] T b7a4572f3919463098951288c395d49d P fc4e99a911a8498fb3fecd21d55491d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:33:08.726282 19742 raft_consensus.cc:374] T b7a4572f3919463098951288c395d49d P 91ec4d92310b4496a89c1eecf66102a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:33:08.726374 19741 raft_consensus.cc:729] T b7a4572f3919463098951288c395d49d P fc4e99a911a8498fb3fecd21d55491d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc4e99a911a8498fb3fecd21d55491d9, State: Initialized, Role: FOLLOWER
I20260811 06:33:08.726374 19742 raft_consensus.cc:729] T b7a4572f3919463098951288c395d49d P 91ec4d92310b4496a89c1eecf66102a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ec4d92310b4496a89c1eecf66102a3, State: Initialized, Role: FOLLOWER
I20260811 06:33:08.727164 19741 consensus_queue.cc:260] T b7a4572f3919463098951288c395d49d P fc4e99a911a8498fb3fecd21d55491d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ec4d92310b4496a89c1eecf66102a3" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37153 } } peers { permanent_uuid: "fc4e99a911a8498fb3fecd21d55491d9" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 37147 } } peers { permanent_uuid: "bb42fe5d26884700bb28ac7c1da33596" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 36329 } }
I20260811 06:33:08.727164 19742 consensus_queue.cc:260] T b7a4572f3919463098951288c395d49d P 91ec4d92310b4496a89c1eecf66102a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ec4d92310b4496a89c1eecf66102a3" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37153 } } peers { permanent_uuid: "fc4e99a911a8498fb3fecd21d55491d9" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 37147 } } peers { permanent_uuid: "bb42fe5d26884700bb28ac7c1da33596" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 36329 } }
I20260811 06:33:08.728880 19743 raft_consensus.cc:348] T b7a4572f3919463098951288c395d49d P bb42fe5d26884700bb28ac7c1da33596 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ec4d92310b4496a89c1eecf66102a3" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37153 } } peers { permanent_uuid: "fc4e99a911a8498fb3fecd21d55491d9" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 37147 } } peers { permanent_uuid: "bb42fe5d26884700bb28ac7c1da33596" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 36329 } }
I20260811 06:33:08.729291 19743 raft_consensus.cc:374] T b7a4572f3919463098951288c395d49d P bb42fe5d26884700bb28ac7c1da33596 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:33:08.729388 19743 raft_consensus.cc:729] T b7a4572f3919463098951288c395d49d P bb42fe5d26884700bb28ac7c1da33596 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb42fe5d26884700bb28ac7c1da33596, State: Initialized, Role: FOLLOWER
I20260811 06:33:08.730137 19743 consensus_queue.cc:260] T b7a4572f3919463098951288c395d49d P bb42fe5d26884700bb28ac7c1da33596 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ec4d92310b4496a89c1eecf66102a3" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37153 } } peers { permanent_uuid: "fc4e99a911a8498fb3fecd21d55491d9" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 37147 } } peers { permanent_uuid: "bb42fe5d26884700bb28ac7c1da33596" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 36329 } }
I20260811 06:33:08.738405 19743 ts_tablet_manager.cc:1428] T b7a4572f3919463098951288c395d49d P bb42fe5d26884700bb28ac7c1da33596: Time spent starting tablet: real 0.018s user 0.014s sys 0.005s
I20260811 06:33:08.738794 19705 heartbeater.cc:503] Master 127.28.239.254:33729 was elected leader, sending a full tablet report...
I20260811 06:33:08.746821 19497 heartbeater.cc:503] Master 127.28.239.254:33729 was elected leader, sending a full tablet report...
I20260811 06:33:08.747215 19741 ts_tablet_manager.cc:1428] T b7a4572f3919463098951288c395d49d P fc4e99a911a8498fb3fecd21d55491d9: Time spent starting tablet: real 0.033s user 0.016s sys 0.015s
I20260811 06:33:08.747664 19742 ts_tablet_manager.cc:1428] T b7a4572f3919463098951288c395d49d P 91ec4d92310b4496a89c1eecf66102a3: Time spent starting tablet: real 0.033s user 0.024s sys 0.008s
W20260811 06:33:08.828142 19708 tablet.cc:2369] T b7a4572f3919463098951288c395d49d P bb42fe5d26884700bb28ac7c1da33596: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:33:08.898846 19755 raft_consensus.cc:482] T b7a4572f3919463098951288c395d49d P bb42fe5d26884700bb28ac7c1da33596 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:33:08.899168 19755 raft_consensus.cc:504] T b7a4572f3919463098951288c395d49d P bb42fe5d26884700bb28ac7c1da33596 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ec4d92310b4496a89c1eecf66102a3" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37153 } } peers { permanent_uuid: "fc4e99a911a8498fb3fecd21d55491d9" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 37147 } } peers { permanent_uuid: "bb42fe5d26884700bb28ac7c1da33596" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 36329 } }
I20260811 06:33:08.900879 19755 leader_election.cc:288] T b7a4572f3919463098951288c395d49d P bb42fe5d26884700bb28ac7c1da33596 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91ec4d92310b4496a89c1eecf66102a3 (127.28.239.194:37153), fc4e99a911a8498fb3fecd21d55491d9 (127.28.239.193:37147)
I20260811 06:33:08.907819 19731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:08.901488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.195:55991 (local address 127.28.239.194:37153)
0811 06:33:08.901724 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0811 06:33:08.901730 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:33:08.901796 (+ 66us) server_negotiation.cc:407] Connection header received
0811 06:33:08.902103 (+ 307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:33:08.902109 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:33:08.902189 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:33:08.902306 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:33:08.902903 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:08.903705 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:08.905433 (+ 1728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:08.906659 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:08.906820 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:08.906902 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 06:33:08.907415 (+ 513us) server_negotiation.cc:299] Negotiation successful
0811 06:33:08.907545 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260811 06:33:08.907862 19710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:08.901258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.194:37153 (local address 127.28.239.195:55991)
0811 06:33:08.901689 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:08.901717 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0811 06:33:08.901905 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:33:08.902360 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:33:08.902364 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:33:08.902420 (+ 56us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:33:08.902748 (+ 328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:33:08.902757 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:33:08.903859 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:33:08.903865 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:33:08.905272 (+ 1407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:33:08.905285 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:33:08.906996 (+ 1711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:33:08.907005 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:33:08.907229 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:08.907285 (+ 56us) client_negotiation.cc:769] Sending connection context
0811 06:33:08.907388 (+ 103us) client_negotiation.cc:240] Negotiation successful
0811 06:33:08.907525 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"spinlock_wait_cycles":68352}
I20260811 06:33:08.908304 19733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:08.901571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.195:42087 (local address 127.28.239.193:37147)
0811 06:33:08.901763 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 06:33:08.901770 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:33:08.902460 (+ 690us) server_negotiation.cc:407] Connection header received
0811 06:33:08.902694 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:33:08.902699 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:33:08.902765 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:33:08.902870 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:33:08.903440 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:08.904163 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:08.906023 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:08.907381 (+ 1358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:08.907568 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:08.907638 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 06:33:08.907920 (+ 282us) server_negotiation.cc:299] Negotiation successful
0811 06:33:08.908051 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 06:33:08.908422 19758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:08.901435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.193:37147 (local address 127.28.239.195:42087)
0811 06:33:08.902344 (+ 909us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:08.902384 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 06:33:08.902523 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:33:08.902888 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:33:08.902892 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:33:08.902915 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:33:08.903212 (+ 297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:33:08.903251 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:33:08.904325 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:33:08.904334 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:33:08.905784 (+ 1450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:33:08.905802 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:33:08.907585 (+ 1783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:33:08.907592 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:33:08.907755 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:08.907778 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 06:33:08.907894 (+ 116us) client_negotiation.cc:240] Negotiation successful
0811 06:33:08.908019 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":744,"thread_start_us":507,"threads_started":1}
I20260811 06:33:08.909008 19414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7a4572f3919463098951288c395d49d" candidate_uuid: "bb42fe5d26884700bb28ac7c1da33596" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ec4d92310b4496a89c1eecf66102a3" is_pre_election: true
I20260811 06:33:08.909433 19211 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7a4572f3919463098951288c395d49d" candidate_uuid: "bb42fe5d26884700bb28ac7c1da33596" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc4e99a911a8498fb3fecd21d55491d9" is_pre_election: true
I20260811 06:33:08.909662 19414 raft_consensus.cc:2454] T b7a4572f3919463098951288c395d49d P 91ec4d92310b4496a89c1eecf66102a3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb42fe5d26884700bb28ac7c1da33596 in term 0.
I20260811 06:33:08.910010 19211 raft_consensus.cc:2454] T b7a4572f3919463098951288c395d49d P fc4e99a911a8498fb3fecd21d55491d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb42fe5d26884700bb28ac7c1da33596 in term 0.
I20260811 06:33:08.910418 19530 leader_election.cc:302] T b7a4572f3919463098951288c395d49d P bb42fe5d26884700bb28ac7c1da33596 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ec4d92310b4496a89c1eecf66102a3, bb42fe5d26884700bb28ac7c1da33596; no voters:
I20260811 06:33:08.910843 19755 raft_consensus.cc:2781] T b7a4572f3919463098951288c395d49d P bb42fe5d26884700bb28ac7c1da33596 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:33:08.910967 19755 raft_consensus.cc:482] T b7a4572f3919463098951288c395d49d P bb42fe5d26884700bb28ac7c1da33596 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:33:08.911077 19755 raft_consensus.cc:3037] T b7a4572f3919463098951288c395d49d P bb42fe5d26884700bb28ac7c1da33596 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:33:08.913357 19755 raft_consensus.cc:504] T b7a4572f3919463098951288c395d49d P bb42fe5d26884700bb28ac7c1da33596 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ec4d92310b4496a89c1eecf66102a3" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37153 } } peers { permanent_uuid: "fc4e99a911a8498fb3fecd21d55491d9" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 37147 } } peers { permanent_uuid: "bb42fe5d26884700bb28ac7c1da33596" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 36329 } }
I20260811 06:33:08.914052 19755 leader_election.cc:288] T b7a4572f3919463098951288c395d49d P bb42fe5d26884700bb28ac7c1da33596 [CANDIDATE]: Term 1 election: Requested vote from peers 91ec4d92310b4496a89c1eecf66102a3 (127.28.239.194:37153), fc4e99a911a8498fb3fecd21d55491d9 (127.28.239.193:37147)
I20260811 06:33:08.914616 19211 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7a4572f3919463098951288c395d49d" candidate_uuid: "bb42fe5d26884700bb28ac7c1da33596" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc4e99a911a8498fb3fecd21d55491d9"
I20260811 06:33:08.914613 19414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7a4572f3919463098951288c395d49d" candidate_uuid: "bb42fe5d26884700bb28ac7c1da33596" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ec4d92310b4496a89c1eecf66102a3"
I20260811 06:33:08.914850 19414 raft_consensus.cc:3037] T b7a4572f3919463098951288c395d49d P 91ec4d92310b4496a89c1eecf66102a3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:33:08.914850 19211 raft_consensus.cc:3037] T b7a4572f3919463098951288c395d49d P fc4e99a911a8498fb3fecd21d55491d9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:33:08.918243 19211 raft_consensus.cc:2454] T b7a4572f3919463098951288c395d49d P fc4e99a911a8498fb3fecd21d55491d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb42fe5d26884700bb28ac7c1da33596 in term 1.
I20260811 06:33:08.918254 19414 raft_consensus.cc:2454] T b7a4572f3919463098951288c395d49d P 91ec4d92310b4496a89c1eecf66102a3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb42fe5d26884700bb28ac7c1da33596 in term 1.
I20260811 06:33:08.918797 19533 leader_election.cc:302] T b7a4572f3919463098951288c395d49d P bb42fe5d26884700bb28ac7c1da33596 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bb42fe5d26884700bb28ac7c1da33596, fc4e99a911a8498fb3fecd21d55491d9; no voters:
I20260811 06:33:08.919118 19755 raft_consensus.cc:2781] T b7a4572f3919463098951288c395d49d P bb42fe5d26884700bb28ac7c1da33596 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:33:08.919620 19755 raft_consensus.cc:686] T b7a4572f3919463098951288c395d49d P bb42fe5d26884700bb28ac7c1da33596 [term 1 LEADER]: Becoming Leader. State: Replica: bb42fe5d26884700bb28ac7c1da33596, State: Running, Role: LEADER
I20260811 06:33:08.920136 19755 consensus_queue.cc:237] T b7a4572f3919463098951288c395d49d P bb42fe5d26884700bb28ac7c1da33596 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ec4d92310b4496a89c1eecf66102a3" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37153 } } peers { permanent_uuid: "fc4e99a911a8498fb3fecd21d55491d9" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 37147 } } peers { permanent_uuid: "bb42fe5d26884700bb28ac7c1da33596" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 36329 } }
I20260811 06:33:08.924885 18973 catalog_manager.cc:5495] T b7a4572f3919463098951288c395d49d P bb42fe5d26884700bb28ac7c1da33596 reported cstate change: term changed from 0 to 1, leader changed from <none> to bb42fe5d26884700bb28ac7c1da33596 (127.28.239.195). New cstate: current_term: 1 leader_uuid: "bb42fe5d26884700bb28ac7c1da33596" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ec4d92310b4496a89c1eecf66102a3" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc4e99a911a8498fb3fecd21d55491d9" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 37147 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb42fe5d26884700bb28ac7c1da33596" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 36329 } health_report { overall_health: HEALTHY } } }
W20260811 06:33:08.941689 19277 tablet.cc:2369] T b7a4572f3919463098951288c395d49d P fc4e99a911a8498fb3fecd21d55491d9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:33:08.959810 19739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:08.953438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54678 (local address 127.28.239.195:36329)
0811 06:33:08.953711 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0811 06:33:08.953719 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:33:08.953833 (+ 114us) server_negotiation.cc:407] Connection header received
0811 06:33:08.954151 (+ 318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:33:08.954156 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:33:08.954248 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:33:08.954434 (+ 186us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:33:08.954893 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:08.955695 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:08.957245 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:08.957785 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:08.957944 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:08.958529 (+ 585us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:33:08.959107 (+ 578us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:33:08.959311 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0811 06:33:08.959370 (+ 59us) server_negotiation.cc:299] Negotiation successful
0811 06:33:08.959465 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114}
I20260811 06:33:08.969705 19414 raft_consensus.cc:1261] T b7a4572f3919463098951288c395d49d P 91ec4d92310b4496a89c1eecf66102a3 [term 1 FOLLOWER]: Refusing update from remote peer bb42fe5d26884700bb28ac7c1da33596: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:33:08.969705 19211 raft_consensus.cc:1261] T b7a4572f3919463098951288c395d49d P fc4e99a911a8498fb3fecd21d55491d9 [term 1 FOLLOWER]: Refusing update from remote peer bb42fe5d26884700bb28ac7c1da33596: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:33:08.970932 19759 consensus_queue.cc:1035] T b7a4572f3919463098951288c395d49d P bb42fe5d26884700bb28ac7c1da33596 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc4e99a911a8498fb3fecd21d55491d9" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 37147 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:33:08.971441 19755 consensus_queue.cc:1035] T b7a4572f3919463098951288c395d49d P bb42fe5d26884700bb28ac7c1da33596 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ec4d92310b4496a89c1eecf66102a3" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:33:08.986114 19765 mvcc.cc:205] Tried to move back new op lower bound from 7317217234807578624 to 7317217234620633088. Current Snapshot: MvccSnapshot[applied={T|T < 7317217234807578624}]
I20260811 06:33:08.987710 19769 mvcc.cc:205] Tried to move back new op lower bound from 7317217234807578624 to 7317217234620633088. Current Snapshot: MvccSnapshot[applied={T|T < 7317217234807578624}]
I20260811 06:33:09.007957 18973 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44222:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:33:09.008494 18973 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44222:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:33:09.012413 18973 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6fb3717db6bf4269ab7e3d2131cd415a: Sending DeleteTablet for 3 replicas of tablet b7a4572f3919463098951288c395d49d
I20260811 06:33:09.013826 19610 tablet_service.cc:1514] Processing DeleteTablet for tablet b7a4572f3919463098951288c395d49d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:33:09 UTC) from {username='kudu', principal='kudu/127.28.239.254@KRBTEST.COM'} at 127.0.0.1:54670
I20260811 06:33:09.013863 19381 tablet_service.cc:1514] Processing DeleteTablet for tablet b7a4572f3919463098951288c395d49d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:33:09 UTC) from {username='kudu', principal='kudu/127.28.239.254@KRBTEST.COM'} at 127.0.0.1:33004
I20260811 06:33:09.014189 19177 tablet_service.cc:1514] Processing DeleteTablet for tablet b7a4572f3919463098951288c395d49d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:33:09 UTC) from {username='kudu', principal='kudu/127.28.239.254@KRBTEST.COM'} at 127.0.0.1:52290
I20260811 06:33:09.015287 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 19078
I20260811 06:33:09.028002 19781 tablet_replica.cc:330] T b7a4572f3919463098951288c395d49d P 91ec4d92310b4496a89c1eecf66102a3: stopping tablet replica
I20260811 06:33:09.028543 19781 raft_consensus.cc:2229] T b7a4572f3919463098951288c395d49d P 91ec4d92310b4496a89c1eecf66102a3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:33:09.029134 19781 raft_consensus.cc:2258] T b7a4572f3919463098951288c395d49d P 91ec4d92310b4496a89c1eecf66102a3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:33:09.030267 19783 tablet_replica.cc:330] T b7a4572f3919463098951288c395d49d P bb42fe5d26884700bb28ac7c1da33596: stopping tablet replica
I20260811 06:33:09.030762 19783 raft_consensus.cc:2229] T b7a4572f3919463098951288c395d49d P bb42fe5d26884700bb28ac7c1da33596 [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:33:09.031677 19783 raft_consensus.cc:2258] T b7a4572f3919463098951288c395d49d P bb42fe5d26884700bb28ac7c1da33596 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:33:09.031731 19781 ts_tablet_manager.cc:1905] T b7a4572f3919463098951288c395d49d P 91ec4d92310b4496a89c1eecf66102a3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:33:09.034150 19783 ts_tablet_manager.cc:1905] T b7a4572f3919463098951288c395d49d P bb42fe5d26884700bb28ac7c1da33596: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:33:09.039734 19781 ts_tablet_manager.cc:1918] T b7a4572f3919463098951288c395d49d P 91ec4d92310b4496a89c1eecf66102a3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:33:09.039947 19781 log.cc:1196] T b7a4572f3919463098951288c395d49d P 91ec4d92310b4496a89c1eecf66102a3: Deleting WAL directory at /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-1/wal/wals/b7a4572f3919463098951288c395d49d
I20260811 06:33:09.040612 19781 ts_tablet_manager.cc:1939] T b7a4572f3919463098951288c395d49d P 91ec4d92310b4496a89c1eecf66102a3: Deleting consensus metadata
I20260811 06:33:09.041769 19783 ts_tablet_manager.cc:1918] T b7a4572f3919463098951288c395d49d P bb42fe5d26884700bb28ac7c1da33596: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:33:09.042007 19783 log.cc:1196] T b7a4572f3919463098951288c395d49d P bb42fe5d26884700bb28ac7c1da33596: Deleting WAL directory at /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-2/wal/wals/b7a4572f3919463098951288c395d49d
I20260811 06:33:09.042623 19783 ts_tablet_manager.cc:1939] T b7a4572f3919463098951288c395d49d P bb42fe5d26884700bb28ac7c1da33596: Deleting consensus metadata
I20260811 06:33:09.043036 18953 catalog_manager.cc:4841] TS 91ec4d92310b4496a89c1eecf66102a3 (127.28.239.194:37153): tablet b7a4572f3919463098951288c395d49d (table test-table [id=28c4fab1e41b4292818028cc1da7a410]) successfully deleted
I20260811 06:33:09.043234 19785 tablet_replica.cc:330] T b7a4572f3919463098951288c395d49d P fc4e99a911a8498fb3fecd21d55491d9: stopping tablet replica
I20260811 06:33:09.043753 19785 raft_consensus.cc:2229] T b7a4572f3919463098951288c395d49d P fc4e99a911a8498fb3fecd21d55491d9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:33:09.044368 19785 raft_consensus.cc:2258] T b7a4572f3919463098951288c395d49d P fc4e99a911a8498fb3fecd21d55491d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:33:09.044569 18954 catalog_manager.cc:4841] TS bb42fe5d26884700bb28ac7c1da33596 (127.28.239.195:36329): tablet b7a4572f3919463098951288c395d49d (table test-table [id=28c4fab1e41b4292818028cc1da7a410]) successfully deleted
I20260811 06:33:09.046950 19785 ts_tablet_manager.cc:1905] T b7a4572f3919463098951288c395d49d P fc4e99a911a8498fb3fecd21d55491d9: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 11 06:33:09 dist-test-slave-gkw7 krb5kdc[18901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429986, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.239.193@KRBTEST.COM
I20260811 06:33:09.053731 19785 ts_tablet_manager.cc:1918] T b7a4572f3919463098951288c395d49d P fc4e99a911a8498fb3fecd21d55491d9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:33:09.053910 19785 log.cc:1196] T b7a4572f3919463098951288c395d49d P fc4e99a911a8498fb3fecd21d55491d9: Deleting WAL directory at /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-0/wal/wals/b7a4572f3919463098951288c395d49d
I20260811 06:33:09.054498 19785 ts_tablet_manager.cc:1939] T b7a4572f3919463098951288c395d49d P fc4e99a911a8498fb3fecd21d55491d9: Deleting consensus metadata
I20260811 06:33:09.056387 18955 catalog_manager.cc:4841] TS fc4e99a911a8498fb3fecd21d55491d9 (127.28.239.193:37147): tablet b7a4572f3919463098951288c395d49d (table test-table [id=28c4fab1e41b4292818028cc1da7a410]) successfully deleted
I20260811 06:33:09.058676 19733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:09.016771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52304 (local address 127.28.239.193:37147)
0811 06:33:09.017003 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0811 06:33:09.017013 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:33:09.033081 (+ 16068us) server_negotiation.cc:407] Connection header received
0811 06:33:09.033756 (+ 675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:33:09.033783 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:33:09.033919 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:33:09.034015 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:33:09.037692 (+ 3677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:09.039316 (+ 1624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:09.043457 (+ 4141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:09.044224 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:09.044373 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:09.050853 (+ 6480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:33:09.050872 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:33:09.050891 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:33:09.050961 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:33:09.055219 (+ 4258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:33:09.056309 (+ 1090us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:33:09.056319 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:33:09.056322 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:33:09.056502 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:33:09.056963 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:33:09.056968 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:33:09.056971 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:33:09.057354 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:33:09.057718 (+ 364us) server_negotiation.cc:1035] Waiting for connection context
0811 06:33:09.058051 (+ 333us) server_negotiation.cc:299] Negotiation successful
0811 06:33:09.058280 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 06:33:09.059312 19267 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:33:09.167133 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 19078
I20260811 06:33:09.196693 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 19283
Aug 11 06:33:09 dist-test-slave-gkw7 krb5kdc[18901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429986, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.239.194@KRBTEST.COM
I20260811 06:33:09.221105 19731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:09.197648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33006 (local address 127.28.239.194:37153)
0811 06:33:09.197863 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 06:33:09.197870 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:33:09.197894 (+ 24us) server_negotiation.cc:407] Connection header received
0811 06:33:09.198086 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:33:09.198093 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:33:09.198203 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:33:09.198335 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:33:09.199728 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:09.200936 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:09.202525 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:09.203249 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:09.203400 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:09.213521 (+ 10121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:33:09.213570 (+ 49us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:33:09.213595 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:33:09.213655 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:33:09.217897 (+ 4242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:33:09.218682 (+ 785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:33:09.218696 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:33:09.218700 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:33:09.218890 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:33:09.219286 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:33:09.219292 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:33:09.219296 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:33:09.219866 (+ 570us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:33:09.220207 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0811 06:33:09.220392 (+ 185us) server_negotiation.cc:299] Negotiation successful
0811 06:33:09.220647 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260811 06:33:09.221987 19491 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:33:09.321358 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 19283
I20260811 06:33:09.349732 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 19505
Aug 11 06:33:09 dist-test-slave-gkw7 krb5kdc[18901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429986, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.239.195@KRBTEST.COM
I20260811 06:33:09.375708 19739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:09.350820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54694 (local address 127.28.239.195:36329)
0811 06:33:09.351079 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0811 06:33:09.351087 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:33:09.351110 (+ 23us) server_negotiation.cc:407] Connection header received
0811 06:33:09.351199 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:33:09.351205 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:33:09.351322 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:33:09.351464 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:33:09.352713 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:09.354101 (+ 1388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:09.355703 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:09.356485 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:09.356652 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:09.367191 (+ 10539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:33:09.367210 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:33:09.367234 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:33:09.367316 (+ 82us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:33:09.372295 (+ 4979us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:33:09.373093 (+ 798us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:33:09.373106 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:33:09.373110 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:33:09.373300 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:33:09.373749 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:33:09.373755 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:33:09.373758 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:33:09.374344 (+ 586us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:33:09.374730 (+ 386us) server_negotiation.cc:1035] Waiting for connection context
0811 06:33:09.374917 (+ 187us) server_negotiation.cc:299] Negotiation successful
0811 06:33:09.375211 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
I20260811 06:33:09.376768 19697 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:33:09.503386 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 19505
I20260811 06:33:09.531754 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 18921
I20260811 06:33:09.532938 19028 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:33:09.648420 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 18921
I20260811 06:33:09.678260 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:33729
--webserver_interface=127.28.239.254
--webserver_port=40223
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab
--principal=kudu/127.28.239.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:42797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.239.254:33729
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:33:10.026495 19796 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:33:10.026854 19796 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:33:10.027017 19796 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:33:10.034364 19796 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:33:10.034516 19796 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:33:10.034627 19796 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:33:10.034731 19796 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:33:10.034816 19796 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:33:10.044818 19796 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:42797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.28.239.254:33729
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab
--principal=kudu/127.28.239.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:33729
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.239.254
--webserver_port=40223
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:33:10.046213 19796 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:33:10.047600 19796 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:33:10.074338 19805 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:33:10.074419 19809 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:33:10.074393 19806 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:33:10.075421 19796 server_base.cc:1034] running on GCE node
I20260811 06:33:10.076354 19796 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:33:10.078193 19796 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:33:10.079455 19796 hybrid_clock.cc:648] HybridClock initialized: now 1786429990079371 us; error 65 us; skew 500 ppm
I20260811 06:33:10.079908 19796 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:33:10 dist-test-slave-gkw7 krb5kdc[18901](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429990, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:33:10.083674 19796 init.cc:377] Logged in from keytab as kudu/127.28.239.254@KRBTEST.COM (short username kudu)
I20260811 06:33:10.085386 19796 webserver.cc:460] Webserver started at http://127.28.239.254:40223/ using document root <none> and password file <none>
I20260811 06:33:10.086220 19796 fs_manager.cc:359] Metadata directory not provided
I20260811 06:33:10.086354 19796 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:33:10.092794 19796 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.003s
I20260811 06:33:10.096033 19820 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:33:10.097903 19796 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 06:33:10.098095 19796 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/master-0/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/master-0/wal
uuid: "6fb3717db6bf4269ab7e3d2131cd415a"
format_stamp: "Formatted at 2026-08-11 06:33:06 on dist-test-slave-gkw7"
server_key: "1111ffd9099c338c4d48ad67d7c515af"
server_key_iv: "712e843f0a62e464bc4221f4edddc5ba"
server_key_version: "encryptionkey@0"
I20260811 06:33:10.099354 19796 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:33:10.121965 19796 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:33:10.126612 19796 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:33:10.126878 19796 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:33:10.148710 19796 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.254:33729
I20260811 06:33:10.148734 19898 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.254:33729 every 8 connection(s)
I20260811 06:33:10.150434 19796 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
I20260811 06:33:10.157812 19900 sys_catalog.cc:263] Verifying existing consensus state
I20260811 06:33:10.158715 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 19796
I20260811 06:33:10.159765 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.193:37147
--local_ip_for_outbound_sockets=127.28.239.193
--tserver_master_addrs=127.28.239.254:33729
--webserver_port=46097
--webserver_interface=127.28.239.193
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.193.keytab
--principal=kudu/127.28.239.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:42797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 06:33:10.163059 19900 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6fb3717db6bf4269ab7e3d2131cd415a: Bootstrap starting.
I20260811 06:33:10.185864 19900 log.cc:824] T 00000000000000000000000000000000 P 6fb3717db6bf4269ab7e3d2131cd415a: Log is configured to *not* fsync() on all Append() calls
I20260811 06:33:10.198213 19900 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6fb3717db6bf4269ab7e3d2131cd415a: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 06:33:10.198777 19900 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6fb3717db6bf4269ab7e3d2131cd415a: Bootstrap complete.
I20260811 06:33:10.208952 19900 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6fb3717db6bf4269ab7e3d2131cd415a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fb3717db6bf4269ab7e3d2131cd415a" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 33729 } }
I20260811 06:33:10.210145 19900 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6fb3717db6bf4269ab7e3d2131cd415a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fb3717db6bf4269ab7e3d2131cd415a, State: Initialized, Role: FOLLOWER
I20260811 06:33:10.210783 19900 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6fb3717db6bf4269ab7e3d2131cd415a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fb3717db6bf4269ab7e3d2131cd415a" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 33729 } }
I20260811 06:33:10.211045 19900 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6fb3717db6bf4269ab7e3d2131cd415a [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:33:10.211184 19900 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6fb3717db6bf4269ab7e3d2131cd415a [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:33:10.211380 19900 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6fb3717db6bf4269ab7e3d2131cd415a [term 1 FOLLOWER]: Advancing to term 2
I20260811 06:33:10.215444 19900 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6fb3717db6bf4269ab7e3d2131cd415a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fb3717db6bf4269ab7e3d2131cd415a" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 33729 } }
I20260811 06:33:10.215992 19900 leader_election.cc:302] T 00000000000000000000000000000000 P 6fb3717db6bf4269ab7e3d2131cd415a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6fb3717db6bf4269ab7e3d2131cd415a; no voters:
I20260811 06:33:10.216615 19900 leader_election.cc:288] T 00000000000000000000000000000000 P 6fb3717db6bf4269ab7e3d2131cd415a [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 06:33:10.216847 19908 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6fb3717db6bf4269ab7e3d2131cd415a [term 2 FOLLOWER]: Leader election won for term 2
I20260811 06:33:10.218206 19908 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6fb3717db6bf4269ab7e3d2131cd415a [term 2 LEADER]: Becoming Leader. State: Replica: 6fb3717db6bf4269ab7e3d2131cd415a, State: Running, Role: LEADER
I20260811 06:33:10.218981 19900 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6fb3717db6bf4269ab7e3d2131cd415a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:33:10.219008 19908 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6fb3717db6bf4269ab7e3d2131cd415a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fb3717db6bf4269ab7e3d2131cd415a" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 33729 } }
I20260811 06:33:10.222332 19910 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6fb3717db6bf4269ab7e3d2131cd415a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6fb3717db6bf4269ab7e3d2131cd415a. Latest consensus state: current_term: 2 leader_uuid: "6fb3717db6bf4269ab7e3d2131cd415a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fb3717db6bf4269ab7e3d2131cd415a" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 33729 } } }
I20260811 06:33:10.222343 19909 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6fb3717db6bf4269ab7e3d2131cd415a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "6fb3717db6bf4269ab7e3d2131cd415a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fb3717db6bf4269ab7e3d2131cd415a" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 33729 } } }
I20260811 06:33:10.222983 19909 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6fb3717db6bf4269ab7e3d2131cd415a [sys.catalog]: This master's current role is: LEADER
I20260811 06:33:10.222990 19910 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6fb3717db6bf4269ab7e3d2131cd415a [sys.catalog]: This master's current role is: LEADER
I20260811 06:33:10.224114 19916 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:33:10.245788 19916 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:33:10.247826 19916 catalog_manager.cc:1259] Loaded cluster ID: 610c580947464c11955b0a775094e958
I20260811 06:33:10.248371 19916 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:33:10.252620 19916 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260811 06:33:10.279510 19916 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260811 06:33:10.290463 19916 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:33:10.291873 19916 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6fb3717db6bf4269ab7e3d2131cd415a: Loaded TSK: 0
I20260811 06:33:10.293027 19916 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 06:33:10.568033 19903 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:33:10.568351 19903 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:33:10.568516 19903 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:33:10.576900 19903 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:33:10.577024 19903 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:33:10.577211 19903 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.193
I20260811 06:33:10.587653 19903 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:42797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.193.keytab
--principal=kudu/127.28.239.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.193:37147
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.239.193
--webserver_port=46097
--tserver_master_addrs=127.28.239.254:33729
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.193
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:33:10.589111 19903 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:33:10.590584 19903 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:33:10.605988 19944 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:33:10.606739 19943 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:33:10.606864 19946 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:33:10.607542 19903 server_base.cc:1034] running on GCE node
I20260811 06:33:10.608306 19903 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:33:10.609560 19903 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:33:10.610823 19903 hybrid_clock.cc:648] HybridClock initialized: now 1786429990610750 us; error 61 us; skew 500 ppm
I20260811 06:33:10.611208 19903 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:33:10 dist-test-slave-gkw7 krb5kdc[18901](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429990, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:33:10.615477 19903 init.cc:377] Logged in from keytab as kudu/127.28.239.193@KRBTEST.COM (short username kudu)
I20260811 06:33:10.617326 19903 webserver.cc:460] Webserver started at http://127.28.239.193:46097/ using document root <none> and password file <none>
I20260811 06:33:10.618290 19903 fs_manager.cc:359] Metadata directory not provided
I20260811 06:33:10.618458 19903 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:33:10.626428 19903 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 06:33:10.630092 19958 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:33:10.631683 19903 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 06:33:10.631887 19903 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-0/wal
uuid: "fc4e99a911a8498fb3fecd21d55491d9"
format_stamp: "Formatted at 2026-08-11 06:33:07 on dist-test-slave-gkw7"
server_key: "87229cf8f4d20f1ce8947f6f5b40d369"
server_key_iv: "1caf3cbbcbf5374eb3fe102526d0da06"
server_key_version: "encryptionkey@0"
I20260811 06:33:10.632928 19903 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:33:10.653337 19903 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:33:10.658252 19903 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:33:10.658553 19903 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:33:10.659608 19903 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:33:10.660988 19903 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:33:10.661088 19903 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:33:10.661230 19903 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:33:10.661283 19903 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:33:10.698117 19903 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.193:37147
I20260811 06:33:10.698200 20137 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.193:37147 every 8 connection(s)
I20260811 06:33:10.699656 19903 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
I20260811 06:33:10.704358 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 19903
I20260811 06:33:10.705058 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.194:37153
--local_ip_for_outbound_sockets=127.28.239.194
--tserver_master_addrs=127.28.239.254:33729
--webserver_port=44943
--webserver_interface=127.28.239.194
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.194.keytab
--principal=kudu/127.28.239.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:42797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 06:33:10 dist-test-slave-gkw7 krb5kdc[18901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429990, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.193@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:33:10.728089 20142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:10.702965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:33729 (local address 127.28.239.193:39469)
0811 06:33:10.703854 (+ 889us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:10.703901 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 06:33:10.705849 (+ 1948us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:33:10.707481 (+ 1632us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:33:10.707489 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:33:10.707990 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:33:10.708734 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:33:10.708759 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:33:10.713099 (+ 4340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:33:10.713106 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:33:10.714551 (+ 1445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:33:10.714565 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:33:10.715688 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:33:10.715697 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:33:10.716286 (+ 589us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:10.717011 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:33:10.717042 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:33:10.719490 (+ 2448us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:33:10.723399 (+ 3909us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:33:10.723411 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:33:10.723427 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:33:10.723879 (+ 452us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:33:10.724375 (+ 496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:33:10.724381 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:33:10.724384 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:33:10.724584 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:33:10.725372 (+ 788us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:33:10.725380 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:33:10.725814 (+ 434us) client_negotiation.cc:769] Sending connection context
0811 06:33:10.726096 (+ 282us) client_negotiation.cc:240] Negotiation successful
0811 06:33:10.726688 (+ 592us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":592,"thread_start_us":355,"threads_started":1}
I20260811 06:33:10.728089 20141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:10.702750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.193:39469 (local address 127.28.239.254:33729)
0811 06:33:10.703413 (+ 663us) server_negotiation.cc:206] Beginning negotiation
0811 06:33:10.703425 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 06:33:10.704043 (+ 618us) server_negotiation.cc:407] Connection header received
0811 06:33:10.706443 (+ 2400us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:33:10.706466 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:33:10.706950 (+ 484us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:33:10.707355 (+ 405us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:33:10.711138 (+ 3783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:10.712899 (+ 1761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:10.714734 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:10.715405 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:10.715641 (+ 236us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:10.719823 (+ 4182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:33:10.719832 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:33:10.719849 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:33:10.719889 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:33:10.723238 (+ 3349us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:33:10.724046 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:33:10.724053 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:33:10.724061 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:33:10.724248 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:33:10.724728 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:33:10.724732 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:33:10.724735 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:33:10.725225 (+ 490us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:33:10.725437 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0811 06:33:10.726332 (+ 895us) server_negotiation.cc:299] Negotiation successful
0811 06:33:10.726731 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":282,"threads_started":1}
I20260811 06:33:10.731177 20139 heartbeater.cc:347] Connected to a master server at 127.28.239.254:33729
I20260811 06:33:10.731621 20139 heartbeater.cc:464] Registering TS with master...
I20260811 06:33:10.732694 20139 heartbeater.cc:511] Master 127.28.239.254:33729 requested a full tablet report, sending...
I20260811 06:33:10.735219 19846 ts_manager.cc:194] Registered new tserver with Master: fc4e99a911a8498fb3fecd21d55491d9 (127.28.239.193:37147)
I20260811 06:33:10.736779 19846 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.239.193@KRBTEST.COM'} at 127.28.239.193:39469
W20260811 06:33:11.082795 20143 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:33:11.083144 20143 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:33:11.083266 20143 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:33:11.091495 20143 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:33:11.091621 20143 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:33:11.091735 20143 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.194
I20260811 06:33:11.101857 20143 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:42797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.194.keytab
--principal=kudu/127.28.239.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.194:37153
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.239.194
--webserver_port=44943
--tserver_master_addrs=127.28.239.254:33729
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.194
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:33:11.103343 20143 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:33:11.104846 20143 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:33:11.118750 20153 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:33:11.118921 20151 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:33:11.118798 20156 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:33:11.119705 20143 server_base.cc:1034] running on GCE node
I20260811 06:33:11.120512 20143 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:33:11.121752 20143 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:33:11.123036 20143 hybrid_clock.cc:648] HybridClock initialized: now 1786429991122994 us; error 53 us; skew 500 ppm
I20260811 06:33:11.123427 20143 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:33:11 dist-test-slave-gkw7 krb5kdc[18901](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429991, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:33:11.127295 20143 init.cc:377] Logged in from keytab as kudu/127.28.239.194@KRBTEST.COM (short username kudu)
I20260811 06:33:11.128959 20143 webserver.cc:460] Webserver started at http://127.28.239.194:44943/ using document root <none> and password file <none>
I20260811 06:33:11.130023 20143 fs_manager.cc:359] Metadata directory not provided
I20260811 06:33:11.130167 20143 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:33:11.136509 20143 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 06:33:11.139911 20169 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:33:11.142037 20143 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.001s
I20260811 06:33:11.142256 20143 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-1/wal
uuid: "91ec4d92310b4496a89c1eecf66102a3"
format_stamp: "Formatted at 2026-08-11 06:33:07 on dist-test-slave-gkw7"
server_key: "37f6f06d05194a49812e2141c9e9a675"
server_key_iv: "8b0ff52b36eeddcaff3c89b29eb22656"
server_key_version: "encryptionkey@0"
I20260811 06:33:11.143863 20143 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:33:11.171916 20143 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:33:11.177093 20143 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:33:11.177433 20143 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:33:11.178802 20143 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:33:11.180420 20143 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:33:11.180562 20143 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:33:11.180729 20143 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:33:11.180832 20143 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:33:11.218315 20143 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.194:37153
I20260811 06:33:11.218376 20348 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.194:37153 every 8 connection(s)
I20260811 06:33:11.220309 20143 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
I20260811 06:33:11.229976 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 20143
I20260811 06:33:11.230770 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.195:36329
--local_ip_for_outbound_sockets=127.28.239.195
--tserver_master_addrs=127.28.239.254:33729
--webserver_port=33353
--webserver_interface=127.28.239.195
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.195.keytab
--principal=kudu/127.28.239.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:42797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 06:33:11 dist-test-slave-gkw7 krb5kdc[18901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429991, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.194@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:33:11.244750 20141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:11.223152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.194:39423 (local address 127.28.239.254:33729)
0811 06:33:11.223388 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0811 06:33:11.223395 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:33:11.224756 (+ 1361us) server_negotiation.cc:407] Connection header received
0811 06:33:11.226716 (+ 1960us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:33:11.226721 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:33:11.226820 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:33:11.226916 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:33:11.228431 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:11.229297 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:11.231858 (+ 2561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:11.232298 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:11.232400 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:11.237396 (+ 4996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:33:11.237402 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:33:11.237406 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:33:11.237446 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:33:11.240658 (+ 3212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:33:11.241705 (+ 1047us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:33:11.241709 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:33:11.241712 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:33:11.241880 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:33:11.242497 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:33:11.242505 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:33:11.242511 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:33:11.242996 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:33:11.243245 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0811 06:33:11.243996 (+ 751us) server_negotiation.cc:299] Negotiation successful
0811 06:33:11.244289 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 06:33:11.245453 20353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:11.223803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:33729 (local address 127.28.239.194:39423)
0811 06:33:11.224573 (+ 770us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:11.224612 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 06:33:11.226427 (+ 1815us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:33:11.227135 (+ 708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:33:11.227143 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:33:11.227673 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:33:11.228264 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:33:11.228284 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:33:11.229498 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:33:11.229502 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:33:11.231694 (+ 2192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:33:11.231713 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:33:11.232553 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:33:11.232562 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:33:11.232903 (+ 341us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:11.233771 (+ 868us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:33:11.233805 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:33:11.237042 (+ 3237us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:33:11.240828 (+ 3786us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:33:11.240840 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:33:11.240861 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:33:11.241571 (+ 710us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:33:11.242004 (+ 433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:33:11.242012 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:33:11.242018 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:33:11.242307 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:33:11.243207 (+ 900us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:33:11.243232 (+ 25us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:33:11.243575 (+ 343us) client_negotiation.cc:769] Sending connection context
0811 06:33:11.243820 (+ 245us) client_negotiation.cc:240] Negotiation successful
0811 06:33:11.244259 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":316,"threads_started":1}
I20260811 06:33:11.247190 20350 heartbeater.cc:347] Connected to a master server at 127.28.239.254:33729
I20260811 06:33:11.247576 20350 heartbeater.cc:464] Registering TS with master...
I20260811 06:33:11.248555 20350 heartbeater.cc:511] Master 127.28.239.254:33729 requested a full tablet report, sending...
I20260811 06:33:11.250284 19846 ts_manager.cc:194] Registered new tserver with Master: 91ec4d92310b4496a89c1eecf66102a3 (127.28.239.194:37153)
I20260811 06:33:11.251137 19846 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.239.194@KRBTEST.COM'} at 127.28.239.194:39423
W20260811 06:33:11.594719 20354 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:33:11.595021 20354 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:33:11.595194 20354 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:33:11.603119 20354 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:33:11.603238 20354 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:33:11.603420 20354 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.195
I20260811 06:33:11.614013 20354 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:42797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.195.keytab
--principal=kudu/127.28.239.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.195:36329
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.239.195
--webserver_port=33353
--tserver_master_addrs=127.28.239.254:33729
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.195
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:33:11.615399 20354 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:33:11.616793 20354 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:33:11.631415 20361 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:33:11.632442 20365 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:33:11.632805 20362 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:33:11.633810 20354 server_base.cc:1034] running on GCE node
I20260811 06:33:11.634495 20354 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:33:11.635483 20354 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:33:11.636685 20354 hybrid_clock.cc:648] HybridClock initialized: now 1786429991636611 us; error 66 us; skew 500 ppm
I20260811 06:33:11.637068 20354 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:33:11 dist-test-slave-gkw7 krb5kdc[18901](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429991, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:33:11.640460 20354 init.cc:377] Logged in from keytab as kudu/127.28.239.195@KRBTEST.COM (short username kudu)
I20260811 06:33:11.642114 20354 webserver.cc:460] Webserver started at http://127.28.239.195:33353/ using document root <none> and password file <none>
I20260811 06:33:11.642819 20354 fs_manager.cc:359] Metadata directory not provided
I20260811 06:33:11.642937 20354 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:33:11.648638 20354 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 06:33:11.651636 20374 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:33:11.653040 20354 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 06:33:11.653234 20354 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-2/wal
uuid: "bb42fe5d26884700bb28ac7c1da33596"
format_stamp: "Formatted at 2026-08-11 06:33:08 on dist-test-slave-gkw7"
server_key: "984b476d0ce85290bf9011a0ea33b0b3"
server_key_iv: "e038ec5b7d17787c35cc14ff98ca44b3"
server_key_version: "encryptionkey@0"
I20260811 06:33:11.654359 20354 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:33:11.687642 20354 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:33:11.692003 20354 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:33:11.692323 20354 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:33:11.693395 20354 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:33:11.694864 20354 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:33:11.694999 20354 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:33:11.695132 20354 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:33:11.695245 20354 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:33:11.732401 20354 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.195:36329
I20260811 06:33:11.732465 20565 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.195:36329 every 8 connection(s)
I20260811 06:33:11.734149 20354 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
I20260811 06:33:11.736218 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 20354
I20260811 06:33:11.740420 20139 heartbeater.cc:503] Master 127.28.239.254:33729 was elected leader, sending a full tablet report...
Aug 11 06:33:11 dist-test-slave-gkw7 krb5kdc[18901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429991, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.195@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:33:11.754633 20569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:11.737515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44234 (local address 127.28.239.254:33729)
0811 06:33:11.738574 (+ 1059us) server_negotiation.cc:206] Beginning negotiation
0811 06:33:11.738581 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:33:11.738602 (+ 21us) server_negotiation.cc:407] Connection header received
0811 06:33:11.738656 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:33:11.738660 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:33:11.738720 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:33:11.738843 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:33:11.740901 (+ 2058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:11.742163 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:11.743825 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:11.744268 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:11.744416 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:11.747349 (+ 2933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:33:11.747355 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:33:11.747362 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:33:11.747395 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:33:11.751088 (+ 3693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:33:11.751824 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:33:11.751864 (+ 40us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:33:11.751867 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:33:11.751992 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:33:11.752901 (+ 909us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:33:11.752907 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:33:11.752912 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:33:11.753217 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:33:11.753580 (+ 363us) server_negotiation.cc:1035] Waiting for connection context
0811 06:33:11.753940 (+ 360us) server_negotiation.cc:299] Negotiation successful
0811 06:33:11.754242 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":915,"thread_start_us":757,"threads_started":1}
I20260811 06:33:11.758582 20141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:11.737374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.195:51149 (local address 127.28.239.254:33729)
0811 06:33:11.737766 (+ 392us) server_negotiation.cc:206] Beginning negotiation
0811 06:33:11.737777 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 06:33:11.739320 (+ 1543us) server_negotiation.cc:407] Connection header received
0811 06:33:11.742744 (+ 3424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:33:11.742749 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:33:11.742831 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:33:11.742934 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:33:11.744491 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:11.745277 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:11.747122 (+ 1845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:11.747646 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:11.747803 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:11.752166 (+ 4363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:33:11.752172 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:33:11.752176 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:33:11.752212 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:33:11.755220 (+ 3008us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:33:11.756014 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:33:11.756018 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:33:11.756020 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:33:11.756236 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:33:11.756749 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:33:11.756753 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:33:11.756755 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:33:11.757015 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:33:11.757204 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0811 06:33:11.758136 (+ 932us) server_negotiation.cc:299] Negotiation successful
0811 06:33:11.758341 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260811 06:33:11.759644 20570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:11.738134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:33729 (local address 127.28.239.195:51149)
0811 06:33:11.739113 (+ 979us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:11.739168 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 06:33:11.742417 (+ 3249us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:33:11.743164 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:33:11.743174 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:33:11.743723 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:33:11.744318 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:33:11.744336 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:33:11.745466 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:33:11.745470 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:33:11.746972 (+ 1502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:33:11.746986 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:33:11.747995 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:33:11.748006 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:33:11.748402 (+ 396us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:11.749133 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:33:11.749164 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:33:11.751944 (+ 2780us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:33:11.755357 (+ 3413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:33:11.755368 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:33:11.755385 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:33:11.755849 (+ 464us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:33:11.756366 (+ 517us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:33:11.756372 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:33:11.756379 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:33:11.756631 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:33:11.757222 (+ 591us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:33:11.757230 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:33:11.757696 (+ 466us) client_negotiation.cc:769] Sending connection context
0811 06:33:11.757944 (+ 248us) client_negotiation.cc:240] Negotiation successful
0811 06:33:11.758404 (+ 460us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":650,"thread_start_us":388,"threads_started":1}
I20260811 06:33:11.761052 20566 heartbeater.cc:347] Connected to a master server at 127.28.239.254:33729
I20260811 06:33:11.761356 20566 heartbeater.cc:464] Registering TS with master...
I20260811 06:33:11.762179 20566 heartbeater.cc:511] Master 127.28.239.254:33729 requested a full tablet report, sending...
I20260811 06:33:11.763655 19846 ts_manager.cc:194] Registered new tserver with Master: bb42fe5d26884700bb28ac7c1da33596 (127.28.239.195:36329)
I20260811 06:33:11.764511 19846 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.239.195@KRBTEST.COM'} at 127.28.239.195:51149
I20260811 06:33:11.769941 29631 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 06:33:11.776818 20141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:11.770927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44238 (local address 127.28.239.254:33729)
0811 06:33:11.771099 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 06:33:11.771103 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:33:11.771665 (+ 562us) server_negotiation.cc:407] Connection header received
0811 06:33:11.771852 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:33:11.771856 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:33:11.771989 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:33:11.772123 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:33:11.772555 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:11.773312 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:11.774795 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:11.775254 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:11.775352 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:11.775809 (+ 457us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:33:11.776323 (+ 514us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:33:11.776433 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 06:33:11.776549 (+ 116us) server_negotiation.cc:299] Negotiation successful
0811 06:33:11.776620 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 06:33:11.777858 19846 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:44238:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:33:11.785684 19846 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:33:11.824406 20588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:11.798360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.195:36329 (local address 127.0.0.1:54710)
0811 06:33:11.810734 (+ 12374us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:11.810813 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0811 06:33:11.811126 (+ 313us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:33:11.815535 (+ 4409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:33:11.815548 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:33:11.815601 (+ 53us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:33:11.816184 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:33:11.816203 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:33:11.818666 (+ 2463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:33:11.818671 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:33:11.821351 (+ 2680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:33:11.821371 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:33:11.823311 (+ 1940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:33:11.823318 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:33:11.823504 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:11.823558 (+ 54us) client_negotiation.cc:769] Sending connection context
0811 06:33:11.823734 (+ 176us) client_negotiation.cc:240] Negotiation successful
0811 06:33:11.823951 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11966,"mutex_wait_us":4,"thread_start_us":18265,"threads_started":1}
I20260811 06:33:11.825227 20589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:11.798985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54710 (local address 127.28.239.195:36329)
0811 06:33:11.814051 (+ 15066us) server_negotiation.cc:206] Beginning negotiation
0811 06:33:11.814066 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 06:33:11.814121 (+ 55us) server_negotiation.cc:407] Connection header received
0811 06:33:11.814271 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:33:11.814284 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:33:11.814669 (+ 385us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:33:11.814936 (+ 267us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:33:11.816479 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:11.818487 (+ 2008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:11.821611 (+ 3124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:11.823097 (+ 1486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:11.823319 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:11.823524 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0811 06:33:11.823771 (+ 247us) server_negotiation.cc:299] Negotiation successful
0811 06:33:11.823948 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14791,"thread_start_us":16214,"threads_started":1}
I20260811 06:33:11.826386 20584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:11.799102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.194:37153 (local address 127.0.0.1:33022)
0811 06:33:11.816925 (+ 17823us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:11.816944 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 06:33:11.817074 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:33:11.817797 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:33:11.817802 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:33:11.817828 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:33:11.818197 (+ 369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:33:11.818206 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:33:11.820179 (+ 1973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:33:11.820185 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:33:11.822662 (+ 2477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:33:11.822679 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:33:11.825382 (+ 2703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:33:11.825389 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:33:11.825632 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:11.825733 (+ 101us) client_negotiation.cc:769] Sending connection context
0811 06:33:11.825841 (+ 108us) client_negotiation.cc:240] Negotiation successful
0811 06:33:11.825991 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17668,"mutex_wait_us":159,"thread_start_us":10585,"threads_started":1}
I20260811 06:33:11.827258 20593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:11.799759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33022 (local address 127.28.239.194:37153)
0811 06:33:11.815426 (+ 15667us) server_negotiation.cc:206] Beginning negotiation
0811 06:33:11.815437 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 06:33:11.817229 (+ 1792us) server_negotiation.cc:407] Connection header received
0811 06:33:11.817338 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:33:11.817345 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:33:11.817611 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:33:11.817814 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:33:11.818389 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:11.820014 (+ 1625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:11.822877 (+ 2863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:11.824769 (+ 1892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:11.824988 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:11.825217 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0811 06:33:11.826665 (+ 1448us) server_negotiation.cc:299] Negotiation successful
0811 06:33:11.826876 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15477,"thread_start_us":15324,"threads_started":1}
I20260811 06:33:11.829058 20590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:11.798669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.193:37147 (local address 127.0.0.1:52312)
0811 06:33:11.811801 (+ 13132us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:11.811827 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 06:33:11.811970 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:33:11.816893 (+ 4923us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:33:11.816902 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:33:11.816938 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:33:11.817401 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:33:11.817416 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:33:11.819580 (+ 2164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:33:11.819610 (+ 30us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:33:11.822013 (+ 2403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:33:11.822027 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:33:11.825417 (+ 3390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:33:11.825426 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:33:11.825651 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:11.828371 (+ 2720us) client_negotiation.cc:769] Sending connection context
0811 06:33:11.828490 (+ 119us) client_negotiation.cc:240] Negotiation successful
0811 06:33:11.828649 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12945,"mutex_wait_us":2632,"thread_start_us":11108,"threads_started":1}
I20260811 06:33:11.829212 20592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:11.799329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52312 (local address 127.28.239.193:37147)
0811 06:33:11.816003 (+ 16674us) server_negotiation.cc:206] Beginning negotiation
0811 06:33:11.816009 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:33:11.816037 (+ 28us) server_negotiation.cc:407] Connection header received
0811 06:33:11.816123 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:33:11.816129 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:33:11.816314 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:33:11.816581 (+ 267us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:33:11.817957 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:11.819425 (+ 1468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:11.822205 (+ 2780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:11.823646 (+ 1441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:11.823877 (+ 231us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:11.825901 (+ 2024us) server_negotiation.cc:1035] Waiting for connection context
0811 06:33:11.828561 (+ 2660us) server_negotiation.cc:299] Negotiation successful
0811 06:33:11.828776 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16504,"thread_start_us":16304,"threads_started":1}
I20260811 06:33:11.829960 20451 tablet_service.cc:1467] Processing CreateTablet for tablet f076fbdcef4f40dda87c8d4ca15f1f42 (DEFAULT_TABLE table=test-table [id=d185bc854708415b9cb4cc93d29ba010]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:33:11.831110 20236 tablet_service.cc:1467] Processing CreateTablet for tablet f076fbdcef4f40dda87c8d4ca15f1f42 (DEFAULT_TABLE table=test-table [id=d185bc854708415b9cb4cc93d29ba010]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:33:11.831813 20451 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f076fbdcef4f40dda87c8d4ca15f1f42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:33:11.832268 20236 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f076fbdcef4f40dda87c8d4ca15f1f42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:33:11.833103 20042 tablet_service.cc:1467] Processing CreateTablet for tablet f076fbdcef4f40dda87c8d4ca15f1f42 (DEFAULT_TABLE table=test-table [id=d185bc854708415b9cb4cc93d29ba010]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:33:11.834578 20042 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f076fbdcef4f40dda87c8d4ca15f1f42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:33:11.844965 20594 tablet_bootstrap.cc:492] T f076fbdcef4f40dda87c8d4ca15f1f42 P 91ec4d92310b4496a89c1eecf66102a3: Bootstrap starting.
I20260811 06:33:11.849046 20596 tablet_bootstrap.cc:492] T f076fbdcef4f40dda87c8d4ca15f1f42 P bb42fe5d26884700bb28ac7c1da33596: Bootstrap starting.
I20260811 06:33:11.849869 20594 tablet_bootstrap.cc:654] T f076fbdcef4f40dda87c8d4ca15f1f42 P 91ec4d92310b4496a89c1eecf66102a3: Neither blocks nor log segments found. Creating new log.
I20260811 06:33:11.851234 20598 tablet_bootstrap.cc:492] T f076fbdcef4f40dda87c8d4ca15f1f42 P fc4e99a911a8498fb3fecd21d55491d9: Bootstrap starting.
I20260811 06:33:11.851265 20594 log.cc:824] T f076fbdcef4f40dda87c8d4ca15f1f42 P 91ec4d92310b4496a89c1eecf66102a3: Log is configured to *not* fsync() on all Append() calls
I20260811 06:33:11.853224 20596 tablet_bootstrap.cc:654] T f076fbdcef4f40dda87c8d4ca15f1f42 P bb42fe5d26884700bb28ac7c1da33596: Neither blocks nor log segments found. Creating new log.
I20260811 06:33:11.854465 20596 log.cc:824] T f076fbdcef4f40dda87c8d4ca15f1f42 P bb42fe5d26884700bb28ac7c1da33596: Log is configured to *not* fsync() on all Append() calls
I20260811 06:33:11.854734 20598 tablet_bootstrap.cc:654] T f076fbdcef4f40dda87c8d4ca15f1f42 P fc4e99a911a8498fb3fecd21d55491d9: Neither blocks nor log segments found. Creating new log.
I20260811 06:33:11.856117 20598 log.cc:824] T f076fbdcef4f40dda87c8d4ca15f1f42 P fc4e99a911a8498fb3fecd21d55491d9: Log is configured to *not* fsync() on all Append() calls
I20260811 06:33:11.856467 20594 tablet_bootstrap.cc:492] T f076fbdcef4f40dda87c8d4ca15f1f42 P 91ec4d92310b4496a89c1eecf66102a3: No bootstrap required, opened a new log
I20260811 06:33:11.856769 20594 ts_tablet_manager.cc:1397] T f076fbdcef4f40dda87c8d4ca15f1f42 P 91ec4d92310b4496a89c1eecf66102a3: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.005s
I20260811 06:33:11.858273 20596 tablet_bootstrap.cc:492] T f076fbdcef4f40dda87c8d4ca15f1f42 P bb42fe5d26884700bb28ac7c1da33596: No bootstrap required, opened a new log
I20260811 06:33:11.858590 20596 ts_tablet_manager.cc:1397] T f076fbdcef4f40dda87c8d4ca15f1f42 P bb42fe5d26884700bb28ac7c1da33596: Time spent bootstrapping tablet: real 0.010s user 0.009s sys 0.000s
I20260811 06:33:11.859627 20598 tablet_bootstrap.cc:492] T f076fbdcef4f40dda87c8d4ca15f1f42 P fc4e99a911a8498fb3fecd21d55491d9: No bootstrap required, opened a new log
I20260811 06:33:11.859918 20598 ts_tablet_manager.cc:1397] T f076fbdcef4f40dda87c8d4ca15f1f42 P fc4e99a911a8498fb3fecd21d55491d9: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.006s
I20260811 06:33:11.867329 20598 raft_consensus.cc:348] T f076fbdcef4f40dda87c8d4ca15f1f42 P fc4e99a911a8498fb3fecd21d55491d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb42fe5d26884700bb28ac7c1da33596" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 36329 } } peers { permanent_uuid: "fc4e99a911a8498fb3fecd21d55491d9" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 37147 } } peers { permanent_uuid: "91ec4d92310b4496a89c1eecf66102a3" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37153 } }
I20260811 06:33:11.867774 20598 raft_consensus.cc:374] T f076fbdcef4f40dda87c8d4ca15f1f42 P fc4e99a911a8498fb3fecd21d55491d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:33:11.867884 20598 raft_consensus.cc:729] T f076fbdcef4f40dda87c8d4ca15f1f42 P fc4e99a911a8498fb3fecd21d55491d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc4e99a911a8498fb3fecd21d55491d9, State: Initialized, Role: FOLLOWER
I20260811 06:33:11.868271 20594 raft_consensus.cc:348] T f076fbdcef4f40dda87c8d4ca15f1f42 P 91ec4d92310b4496a89c1eecf66102a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb42fe5d26884700bb28ac7c1da33596" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 36329 } } peers { permanent_uuid: "fc4e99a911a8498fb3fecd21d55491d9" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 37147 } } peers { permanent_uuid: "91ec4d92310b4496a89c1eecf66102a3" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37153 } }
I20260811 06:33:11.868893 20594 raft_consensus.cc:374] T f076fbdcef4f40dda87c8d4ca15f1f42 P 91ec4d92310b4496a89c1eecf66102a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:33:11.868994 20594 raft_consensus.cc:729] T f076fbdcef4f40dda87c8d4ca15f1f42 P 91ec4d92310b4496a89c1eecf66102a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ec4d92310b4496a89c1eecf66102a3, State: Initialized, Role: FOLLOWER
I20260811 06:33:11.868884 20598 consensus_queue.cc:260] T f076fbdcef4f40dda87c8d4ca15f1f42 P fc4e99a911a8498fb3fecd21d55491d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb42fe5d26884700bb28ac7c1da33596" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 36329 } } peers { permanent_uuid: "fc4e99a911a8498fb3fecd21d55491d9" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 37147 } } peers { permanent_uuid: "91ec4d92310b4496a89c1eecf66102a3" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37153 } }
I20260811 06:33:11.869400 20596 raft_consensus.cc:348] T f076fbdcef4f40dda87c8d4ca15f1f42 P bb42fe5d26884700bb28ac7c1da33596 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb42fe5d26884700bb28ac7c1da33596" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 36329 } } peers { permanent_uuid: "fc4e99a911a8498fb3fecd21d55491d9" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 37147 } } peers { permanent_uuid: "91ec4d92310b4496a89c1eecf66102a3" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37153 } }
I20260811 06:33:11.869623 20594 consensus_queue.cc:260] T f076fbdcef4f40dda87c8d4ca15f1f42 P 91ec4d92310b4496a89c1eecf66102a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb42fe5d26884700bb28ac7c1da33596" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 36329 } } peers { permanent_uuid: "fc4e99a911a8498fb3fecd21d55491d9" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 37147 } } peers { permanent_uuid: "91ec4d92310b4496a89c1eecf66102a3" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37153 } }
I20260811 06:33:11.869889 20596 raft_consensus.cc:374] T f076fbdcef4f40dda87c8d4ca15f1f42 P bb42fe5d26884700bb28ac7c1da33596 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:33:11.869995 20596 raft_consensus.cc:729] T f076fbdcef4f40dda87c8d4ca15f1f42 P bb42fe5d26884700bb28ac7c1da33596 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb42fe5d26884700bb28ac7c1da33596, State: Initialized, Role: FOLLOWER
I20260811 06:33:11.870724 20596 consensus_queue.cc:260] T f076fbdcef4f40dda87c8d4ca15f1f42 P bb42fe5d26884700bb28ac7c1da33596 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb42fe5d26884700bb28ac7c1da33596" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 36329 } } peers { permanent_uuid: "fc4e99a911a8498fb3fecd21d55491d9" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 37147 } } peers { permanent_uuid: "91ec4d92310b4496a89c1eecf66102a3" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37153 } }
I20260811 06:33:11.871567 20598 ts_tablet_manager.cc:1428] T f076fbdcef4f40dda87c8d4ca15f1f42 P fc4e99a911a8498fb3fecd21d55491d9: Time spent starting tablet: real 0.011s user 0.011s sys 0.000s
I20260811 06:33:11.873155 20350 heartbeater.cc:503] Master 127.28.239.254:33729 was elected leader, sending a full tablet report...
I20260811 06:33:11.874153 20594 ts_tablet_manager.cc:1428] T f076fbdcef4f40dda87c8d4ca15f1f42 P 91ec4d92310b4496a89c1eecf66102a3: Time spent starting tablet: real 0.017s user 0.012s sys 0.006s
I20260811 06:33:11.875365 20566 heartbeater.cc:503] Master 127.28.239.254:33729 was elected leader, sending a full tablet report...
I20260811 06:33:11.876045 20596 ts_tablet_manager.cc:1428] T f076fbdcef4f40dda87c8d4ca15f1f42 P bb42fe5d26884700bb28ac7c1da33596: Time spent starting tablet: real 0.017s user 0.009s sys 0.007s
W20260811 06:33:11.952124 20140 tablet.cc:2369] T f076fbdcef4f40dda87c8d4ca15f1f42 P fc4e99a911a8498fb3fecd21d55491d9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 06:33:11.972631 20352 tablet.cc:2369] T f076fbdcef4f40dda87c8d4ca15f1f42 P 91ec4d92310b4496a89c1eecf66102a3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 06:33:11.985947 20567 tablet.cc:2369] T f076fbdcef4f40dda87c8d4ca15f1f42 P bb42fe5d26884700bb28ac7c1da33596: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:33:11.992460 20603 raft_consensus.cc:482] T f076fbdcef4f40dda87c8d4ca15f1f42 P fc4e99a911a8498fb3fecd21d55491d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:33:11.992851 20603 raft_consensus.cc:504] T f076fbdcef4f40dda87c8d4ca15f1f42 P fc4e99a911a8498fb3fecd21d55491d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb42fe5d26884700bb28ac7c1da33596" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 36329 } } peers { permanent_uuid: "fc4e99a911a8498fb3fecd21d55491d9" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 37147 } } peers { permanent_uuid: "91ec4d92310b4496a89c1eecf66102a3" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37153 } }
I20260811 06:33:11.994982 20603 leader_election.cc:288] T f076fbdcef4f40dda87c8d4ca15f1f42 P fc4e99a911a8498fb3fecd21d55491d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb42fe5d26884700bb28ac7c1da33596 (127.28.239.195:36329), 91ec4d92310b4496a89c1eecf66102a3 (127.28.239.194:37153)
I20260811 06:33:12.004904 20589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:11.995474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.193:45997 (local address 127.28.239.195:36329)
0811 06:33:11.995724 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0811 06:33:11.995731 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:33:11.996562 (+ 831us) server_negotiation.cc:407] Connection header received
0811 06:33:11.996803 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:33:11.996809 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:33:11.996895 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:33:11.997003 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:33:11.997950 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:11.999084 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:12.002385 (+ 3301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:12.003697 (+ 1312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:12.003867 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:12.003948 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 06:33:12.004504 (+ 556us) server_negotiation.cc:299] Negotiation successful
0811 06:33:12.004641 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260811 06:33:12.005012 20607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:11.995392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.195:36329 (local address 127.28.239.193:45997)
0811 06:33:11.996421 (+ 1029us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:11.996438 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 06:33:11.996665 (+ 227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:33:11.997045 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:33:11.997092 (+ 47us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:33:11.997111 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:33:11.997769 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:33:11.997780 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:33:11.999291 (+ 1511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:33:11.999297 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:33:12.002206 (+ 2909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:33:12.002216 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:33:12.004006 (+ 1790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:33:12.004011 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:33:12.004314 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:12.004361 (+ 47us) client_negotiation.cc:769] Sending connection context
0811 06:33:12.004442 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 06:33:12.004543 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":918,"mutex_wait_us":347,"spinlock_wait_cycles":41216,"thread_start_us":618,"threads_started":1}
I20260811 06:33:12.005082 20608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:11.995339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.194:37153 (local address 127.28.239.193:60725)
0811 06:33:11.996386 (+ 1047us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:11.996424 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 06:33:11.996618 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:33:11.997045 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:33:11.997050 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:33:11.997099 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:33:11.997764 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:33:11.997780 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:33:11.999292 (+ 1512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:33:11.999297 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:33:12.002198 (+ 2901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:33:12.002216 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:33:12.003986 (+ 1770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:33:12.003994 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:33:12.004313 (+ 319us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:12.004355 (+ 42us) client_negotiation.cc:769] Sending connection context
0811 06:33:12.004473 (+ 118us) client_negotiation.cc:240] Negotiation successful
0811 06:33:12.004607 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":851,"mutex_wait_us":277,"thread_start_us":652,"threads_started":1}
I20260811 06:33:12.005609 20593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:11.995723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.193:60725 (local address 127.28.239.194:37153)
0811 06:33:11.996009 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0811 06:33:11.996017 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:33:11.996557 (+ 540us) server_negotiation.cc:407] Connection header received
0811 06:33:11.996803 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:33:11.996809 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:33:11.996905 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:33:11.997003 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:33:11.997950 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:11.999084 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:12.002386 (+ 3302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:12.003682 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:12.003870 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:12.003948 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 06:33:12.005170 (+ 1222us) server_negotiation.cc:299] Negotiation successful
0811 06:33:12.005339 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 06:33:12.006114 20497 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f076fbdcef4f40dda87c8d4ca15f1f42" candidate_uuid: "fc4e99a911a8498fb3fecd21d55491d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb42fe5d26884700bb28ac7c1da33596" is_pre_election: true
I20260811 06:33:12.006516 20267 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f076fbdcef4f40dda87c8d4ca15f1f42" candidate_uuid: "fc4e99a911a8498fb3fecd21d55491d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ec4d92310b4496a89c1eecf66102a3" is_pre_election: true
I20260811 06:33:12.006651 20497 raft_consensus.cc:2454] T f076fbdcef4f40dda87c8d4ca15f1f42 P bb42fe5d26884700bb28ac7c1da33596 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc4e99a911a8498fb3fecd21d55491d9 in term 0.
I20260811 06:33:12.007004 20267 raft_consensus.cc:2454] T f076fbdcef4f40dda87c8d4ca15f1f42 P 91ec4d92310b4496a89c1eecf66102a3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc4e99a911a8498fb3fecd21d55491d9 in term 0.
I20260811 06:33:12.007411 19961 leader_election.cc:302] T f076fbdcef4f40dda87c8d4ca15f1f42 P fc4e99a911a8498fb3fecd21d55491d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bb42fe5d26884700bb28ac7c1da33596, fc4e99a911a8498fb3fecd21d55491d9; no voters:
I20260811 06:33:12.007829 20603 raft_consensus.cc:2781] T f076fbdcef4f40dda87c8d4ca15f1f42 P fc4e99a911a8498fb3fecd21d55491d9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:33:12.007988 20603 raft_consensus.cc:482] T f076fbdcef4f40dda87c8d4ca15f1f42 P fc4e99a911a8498fb3fecd21d55491d9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:33:12.008100 20603 raft_consensus.cc:3037] T f076fbdcef4f40dda87c8d4ca15f1f42 P fc4e99a911a8498fb3fecd21d55491d9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:33:12.010790 20603 raft_consensus.cc:504] T f076fbdcef4f40dda87c8d4ca15f1f42 P fc4e99a911a8498fb3fecd21d55491d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb42fe5d26884700bb28ac7c1da33596" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 36329 } } peers { permanent_uuid: "fc4e99a911a8498fb3fecd21d55491d9" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 37147 } } peers { permanent_uuid: "91ec4d92310b4496a89c1eecf66102a3" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37153 } }
I20260811 06:33:12.011519 20603 leader_election.cc:288] T f076fbdcef4f40dda87c8d4ca15f1f42 P fc4e99a911a8498fb3fecd21d55491d9 [CANDIDATE]: Term 1 election: Requested vote from peers bb42fe5d26884700bb28ac7c1da33596 (127.28.239.195:36329), 91ec4d92310b4496a89c1eecf66102a3 (127.28.239.194:37153)
I20260811 06:33:12.011888 20497 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f076fbdcef4f40dda87c8d4ca15f1f42" candidate_uuid: "fc4e99a911a8498fb3fecd21d55491d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb42fe5d26884700bb28ac7c1da33596"
I20260811 06:33:12.012027 20267 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f076fbdcef4f40dda87c8d4ca15f1f42" candidate_uuid: "fc4e99a911a8498fb3fecd21d55491d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ec4d92310b4496a89c1eecf66102a3"
I20260811 06:33:12.012140 20497 raft_consensus.cc:3037] T f076fbdcef4f40dda87c8d4ca15f1f42 P bb42fe5d26884700bb28ac7c1da33596 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:33:12.012290 20267 raft_consensus.cc:3037] T f076fbdcef4f40dda87c8d4ca15f1f42 P 91ec4d92310b4496a89c1eecf66102a3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:33:12.014709 20497 raft_consensus.cc:2454] T f076fbdcef4f40dda87c8d4ca15f1f42 P bb42fe5d26884700bb28ac7c1da33596 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc4e99a911a8498fb3fecd21d55491d9 in term 1.
I20260811 06:33:12.014763 20267 raft_consensus.cc:2454] T f076fbdcef4f40dda87c8d4ca15f1f42 P 91ec4d92310b4496a89c1eecf66102a3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc4e99a911a8498fb3fecd21d55491d9 in term 1.
I20260811 06:33:12.015185 19960 leader_election.cc:302] T f076fbdcef4f40dda87c8d4ca15f1f42 P fc4e99a911a8498fb3fecd21d55491d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ec4d92310b4496a89c1eecf66102a3, fc4e99a911a8498fb3fecd21d55491d9; no voters:
I20260811 06:33:12.015533 20603 raft_consensus.cc:2781] T f076fbdcef4f40dda87c8d4ca15f1f42 P fc4e99a911a8498fb3fecd21d55491d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:33:12.016038 20603 raft_consensus.cc:686] T f076fbdcef4f40dda87c8d4ca15f1f42 P fc4e99a911a8498fb3fecd21d55491d9 [term 1 LEADER]: Becoming Leader. State: Replica: fc4e99a911a8498fb3fecd21d55491d9, State: Running, Role: LEADER
I20260811 06:33:12.016534 20603 consensus_queue.cc:237] T f076fbdcef4f40dda87c8d4ca15f1f42 P fc4e99a911a8498fb3fecd21d55491d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb42fe5d26884700bb28ac7c1da33596" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 36329 } } peers { permanent_uuid: "fc4e99a911a8498fb3fecd21d55491d9" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 37147 } } peers { permanent_uuid: "91ec4d92310b4496a89c1eecf66102a3" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37153 } }
I20260811 06:33:12.021452 19845 catalog_manager.cc:5495] T f076fbdcef4f40dda87c8d4ca15f1f42 P fc4e99a911a8498fb3fecd21d55491d9 reported cstate change: term changed from 0 to 1, leader changed from <none> to fc4e99a911a8498fb3fecd21d55491d9 (127.28.239.193). New cstate: current_term: 1 leader_uuid: "fc4e99a911a8498fb3fecd21d55491d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb42fe5d26884700bb28ac7c1da33596" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 36329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc4e99a911a8498fb3fecd21d55491d9" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 37147 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ec4d92310b4496a89c1eecf66102a3" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37153 } health_report { overall_health: UNKNOWN } } }
I20260811 06:33:12.070468 20592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:12.064931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52320 (local address 127.28.239.193:37147)
0811 06:33:12.065143 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 06:33:12.065147 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:33:12.065166 (+ 19us) server_negotiation.cc:407] Connection header received
0811 06:33:12.065354 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:33:12.065358 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:33:12.065444 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:33:12.065655 (+ 211us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:33:12.066022 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:12.066746 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:12.068201 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:12.068627 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:12.068735 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:12.069319 (+ 584us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:33:12.069944 (+ 625us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:33:12.070061 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 06:33:12.070184 (+ 123us) server_negotiation.cc:299] Negotiation successful
0811 06:33:12.070256 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 06:33:12.079118 20497 raft_consensus.cc:1261] T f076fbdcef4f40dda87c8d4ca15f1f42 P bb42fe5d26884700bb28ac7c1da33596 [term 1 FOLLOWER]: Refusing update from remote peer fc4e99a911a8498fb3fecd21d55491d9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:33:12.079133 20267 raft_consensus.cc:1261] T f076fbdcef4f40dda87c8d4ca15f1f42 P 91ec4d92310b4496a89c1eecf66102a3 [term 1 FOLLOWER]: Refusing update from remote peer fc4e99a911a8498fb3fecd21d55491d9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:33:12.080165 20609 consensus_queue.cc:1035] T f076fbdcef4f40dda87c8d4ca15f1f42 P fc4e99a911a8498fb3fecd21d55491d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ec4d92310b4496a89c1eecf66102a3" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:33:12.080528 20603 consensus_queue.cc:1035] T f076fbdcef4f40dda87c8d4ca15f1f42 P fc4e99a911a8498fb3fecd21d55491d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb42fe5d26884700bb28ac7c1da33596" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 36329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:33:12.094455 20614 mvcc.cc:205] Tried to move back new op lower bound from 7317217247545409536 to 7317217247303958528. Current Snapshot: MvccSnapshot[applied={T|T < 7317217247545409536}]
I20260811 06:33:12.097036 20618 mvcc.cc:205] Tried to move back new op lower bound from 7317217247545409536 to 7317217247303958528. Current Snapshot: MvccSnapshot[applied={T|T < 7317217247545409536}]
I20260811 06:33:12.109740 19846 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:44238:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:33:12.110049 19846 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:44238:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:33:12.113919 19846 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6fb3717db6bf4269ab7e3d2131cd415a: Sending DeleteTablet for 3 replicas of tablet f076fbdcef4f40dda87c8d4ca15f1f42
I20260811 06:33:12.115185 20042 tablet_service.cc:1514] Processing DeleteTablet for tablet f076fbdcef4f40dda87c8d4ca15f1f42 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:33:12 UTC) from {username='kudu', principal='kudu/127.28.239.254@KRBTEST.COM'} at 127.0.0.1:52312
I20260811 06:33:12.116195 20635 tablet_replica.cc:330] T f076fbdcef4f40dda87c8d4ca15f1f42 P fc4e99a911a8498fb3fecd21d55491d9: stopping tablet replica
I20260811 06:33:12.116083 20451 tablet_service.cc:1514] Processing DeleteTablet for tablet f076fbdcef4f40dda87c8d4ca15f1f42 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:33:12 UTC) from {username='kudu', principal='kudu/127.28.239.254@KRBTEST.COM'} at 127.0.0.1:54710
I20260811 06:33:12.116668 20635 raft_consensus.cc:2229] T f076fbdcef4f40dda87c8d4ca15f1f42 P fc4e99a911a8498fb3fecd21d55491d9 [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:33:12.117096 20637 tablet_replica.cc:330] T f076fbdcef4f40dda87c8d4ca15f1f42 P bb42fe5d26884700bb28ac7c1da33596: stopping tablet replica
I20260811 06:33:12.117327 20236 tablet_service.cc:1514] Processing DeleteTablet for tablet f076fbdcef4f40dda87c8d4ca15f1f42 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:33:12 UTC) from {username='kudu', principal='kudu/127.28.239.254@KRBTEST.COM'} at 127.0.0.1:33022
I20260811 06:33:12.117452 20635 raft_consensus.cc:2258] T f076fbdcef4f40dda87c8d4ca15f1f42 P fc4e99a911a8498fb3fecd21d55491d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:33:12.117748 20637 raft_consensus.cc:2229] T f076fbdcef4f40dda87c8d4ca15f1f42 P bb42fe5d26884700bb28ac7c1da33596 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:33:12.118278 20637 raft_consensus.cc:2258] T f076fbdcef4f40dda87c8d4ca15f1f42 P bb42fe5d26884700bb28ac7c1da33596 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:33:12.118705 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 19903
I20260811 06:33:12.120455 20637 ts_tablet_manager.cc:1905] T f076fbdcef4f40dda87c8d4ca15f1f42 P bb42fe5d26884700bb28ac7c1da33596: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:33:12.120463 20635 ts_tablet_manager.cc:1905] T f076fbdcef4f40dda87c8d4ca15f1f42 P fc4e99a911a8498fb3fecd21d55491d9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:33:12.127919 20637 ts_tablet_manager.cc:1918] T f076fbdcef4f40dda87c8d4ca15f1f42 P bb42fe5d26884700bb28ac7c1da33596: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:33:12.128109 20637 log.cc:1196] T f076fbdcef4f40dda87c8d4ca15f1f42 P bb42fe5d26884700bb28ac7c1da33596: Deleting WAL directory at /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-2/wal/wals/f076fbdcef4f40dda87c8d4ca15f1f42
I20260811 06:33:12.128749 20637 ts_tablet_manager.cc:1939] T f076fbdcef4f40dda87c8d4ca15f1f42 P bb42fe5d26884700bb28ac7c1da33596: Deleting consensus metadata
I20260811 06:33:12.128890 20635 ts_tablet_manager.cc:1918] T f076fbdcef4f40dda87c8d4ca15f1f42 P fc4e99a911a8498fb3fecd21d55491d9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:33:12.129140 20635 log.cc:1196] T f076fbdcef4f40dda87c8d4ca15f1f42 P fc4e99a911a8498fb3fecd21d55491d9: Deleting WAL directory at /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-0/wal/wals/f076fbdcef4f40dda87c8d4ca15f1f42
I20260811 06:33:12.130937 20635 ts_tablet_manager.cc:1939] T f076fbdcef4f40dda87c8d4ca15f1f42 P fc4e99a911a8498fb3fecd21d55491d9: Deleting consensus metadata
I20260811 06:33:12.130903 19824 catalog_manager.cc:4841] TS bb42fe5d26884700bb28ac7c1da33596 (127.28.239.195:36329): tablet f076fbdcef4f40dda87c8d4ca15f1f42 (table test-table [id=d185bc854708415b9cb4cc93d29ba010]) successfully deleted
I20260811 06:33:12.133059 19825 catalog_manager.cc:4841] TS fc4e99a911a8498fb3fecd21d55491d9 (127.28.239.193:37147): tablet f076fbdcef4f40dda87c8d4ca15f1f42 (table test-table [id=d185bc854708415b9cb4cc93d29ba010]) successfully deleted
I20260811 06:33:12.135454 20638 tablet_replica.cc:330] T f076fbdcef4f40dda87c8d4ca15f1f42 P 91ec4d92310b4496a89c1eecf66102a3: stopping tablet replica
I20260811 06:33:12.135916 20638 raft_consensus.cc:2229] T f076fbdcef4f40dda87c8d4ca15f1f42 P 91ec4d92310b4496a89c1eecf66102a3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:33:12.136512 20638 raft_consensus.cc:2258] T f076fbdcef4f40dda87c8d4ca15f1f42 P 91ec4d92310b4496a89c1eecf66102a3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:33:12.138888 20638 ts_tablet_manager.cc:1905] T f076fbdcef4f40dda87c8d4ca15f1f42 P 91ec4d92310b4496a89c1eecf66102a3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:33:12.139734 20592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:12.119580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52336 (local address 127.28.239.193:37147)
0811 06:33:12.119775 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 06:33:12.119782 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:33:12.119800 (+ 18us) server_negotiation.cc:407] Connection header received
0811 06:33:12.119990 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:33:12.119998 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:33:12.120101 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:33:12.120212 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:33:12.122091 (+ 1879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:12.123590 (+ 1499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:12.125735 (+ 2145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:12.126802 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:12.126933 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:12.131839 (+ 4906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:33:12.131852 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:33:12.131877 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:33:12.131936 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:33:12.136227 (+ 4291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:33:12.137184 (+ 957us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:33:12.137193 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:33:12.137198 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:33:12.137374 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:33:12.137888 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:33:12.137893 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:33:12.137897 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:33:12.138382 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:33:12.138711 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0811 06:33:12.139085 (+ 374us) server_negotiation.cc:299] Negotiation successful
0811 06:33:12.139333 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 06:33:12.140478 20131 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:33:12.147179 20638 ts_tablet_manager.cc:1918] T f076fbdcef4f40dda87c8d4ca15f1f42 P 91ec4d92310b4496a89c1eecf66102a3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:33:12.147372 20638 log.cc:1196] T f076fbdcef4f40dda87c8d4ca15f1f42 P 91ec4d92310b4496a89c1eecf66102a3: Deleting WAL directory at /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-1/wal/wals/f076fbdcef4f40dda87c8d4ca15f1f42
I20260811 06:33:12.148120 20638 ts_tablet_manager.cc:1939] T f076fbdcef4f40dda87c8d4ca15f1f42 P 91ec4d92310b4496a89c1eecf66102a3: Deleting consensus metadata
I20260811 06:33:12.150074 19822 catalog_manager.cc:4841] TS 91ec4d92310b4496a89c1eecf66102a3 (127.28.239.194:37153): tablet f076fbdcef4f40dda87c8d4ca15f1f42 (table test-table [id=d185bc854708415b9cb4cc93d29ba010]) successfully deleted
I20260811 06:33:12.249190 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 19903
I20260811 06:33:12.273712 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 20143
I20260811 06:33:12.288158 20593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:12.274635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33028 (local address 127.28.239.194:37153)
0811 06:33:12.274820 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 06:33:12.274825 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:33:12.274843 (+ 18us) server_negotiation.cc:407] Connection header received
0811 06:33:12.275019 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:33:12.275024 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:33:12.275095 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:33:12.275237 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:33:12.276583 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:12.277389 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:12.278763 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:12.279176 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:12.279274 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:12.282086 (+ 2812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:33:12.282102 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:33:12.282119 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:33:12.282154 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:33:12.285230 (+ 3076us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:33:12.286213 (+ 983us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:33:12.286221 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:33:12.286223 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:33:12.286360 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:33:12.286692 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:33:12.286695 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:33:12.286697 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:33:12.287104 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:33:12.287338 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0811 06:33:12.287597 (+ 259us) server_negotiation.cc:299] Negotiation successful
0811 06:33:12.287763 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 06:33:12.288916 20340 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:33:12.380267 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 20143
I20260811 06:33:12.408138 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 20354
I20260811 06:33:12.423151 20589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:12.408978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54726 (local address 127.28.239.195:36329)
0811 06:33:12.409185 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 06:33:12.409189 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:33:12.409209 (+ 20us) server_negotiation.cc:407] Connection header received
0811 06:33:12.409417 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:33:12.409422 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:33:12.409509 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:33:12.409649 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:33:12.411676 (+ 2027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:12.412385 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:12.413921 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:12.414298 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:12.414392 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:12.417065 (+ 2673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:33:12.417080 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:33:12.417098 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:33:12.417132 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:33:12.420178 (+ 3046us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:33:12.420831 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:33:12.420840 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:33:12.420842 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:33:12.420990 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:33:12.421449 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:33:12.421453 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:33:12.421456 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:33:12.421936 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:33:12.422183 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0811 06:33:12.422481 (+ 298us) server_negotiation.cc:299] Negotiation successful
0811 06:33:12.422667 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 06:33:12.423842 20559 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:33:12.515375 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 20354
I20260811 06:33:12.539825 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 19796
I20260811 06:33:12.540966 19893 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:33:12.656250 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 19796
I20260811 06:33:12.688477 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:33729
--webserver_interface=127.28.239.254
--webserver_port=40223
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab
--principal=kudu/127.28.239.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:42797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.239.254:33729
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:33:13.036398 20646 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:33:13.036697 20646 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:33:13.036823 20646 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:33:13.044131 20646 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:33:13.044260 20646 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:33:13.044374 20646 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:33:13.044474 20646 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:33:13.044565 20646 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:33:13.054982 20646 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:42797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.239.254:33729
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab
--principal=kudu/127.28.239.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:33729
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.239.254
--webserver_port=40223
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:33:13.056528 20646 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:33:13.058002 20646 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:33:13.070545 20659 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:33:13.070652 20653 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:33:13.070600 20656 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:33:13.071696 20646 server_base.cc:1034] running on GCE node
I20260811 06:33:13.072543 20646 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:33:13.074096 20646 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:33:13.075310 20646 hybrid_clock.cc:648] HybridClock initialized: now 1786429993075259 us; error 38 us; skew 500 ppm
I20260811 06:33:13.075733 20646 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:33:13 dist-test-slave-gkw7 krb5kdc[18901](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429993, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:33:13.079088 20646 init.cc:377] Logged in from keytab as kudu/127.28.239.254@KRBTEST.COM (short username kudu)
I20260811 06:33:13.080705 20646 webserver.cc:460] Webserver started at http://127.28.239.254:40223/ using document root <none> and password file <none>
I20260811 06:33:13.081458 20646 fs_manager.cc:359] Metadata directory not provided
I20260811 06:33:13.081627 20646 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:33:13.087333 20646 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 06:33:13.092168 20671 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:33:13.093667 20646 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:33:13.093865 20646 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/master-0/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/master-0/wal
uuid: "6fb3717db6bf4269ab7e3d2131cd415a"
format_stamp: "Formatted at 2026-08-11 06:33:06 on dist-test-slave-gkw7"
server_key: "1111ffd9099c338c4d48ad67d7c515af"
server_key_iv: "712e843f0a62e464bc4221f4edddc5ba"
server_key_version: "encryptionkey@0"
I20260811 06:33:13.094889 20646 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:33:13.128788 20646 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:33:13.133476 20646 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:33:13.133806 20646 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:33:13.156575 20646 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.254:33729
I20260811 06:33:13.156596 20768 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.254:33729 every 8 connection(s)
I20260811 06:33:13.158457 20646 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
I20260811 06:33:13.165714 20770 sys_catalog.cc:263] Verifying existing consensus state
I20260811 06:33:13.168390 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 20646
I20260811 06:33:13.169126 20770 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6fb3717db6bf4269ab7e3d2131cd415a: Bootstrap starting.
I20260811 06:33:13.169332 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.193:37147
--local_ip_for_outbound_sockets=127.28.239.193
--tserver_master_addrs=127.28.239.254:33729
--webserver_port=46097
--webserver_interface=127.28.239.193
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.193.keytab
--principal=kudu/127.28.239.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:42797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 06:33:13.190364 20770 log.cc:824] T 00000000000000000000000000000000 P 6fb3717db6bf4269ab7e3d2131cd415a: Log is configured to *not* fsync() on all Append() calls
I20260811 06:33:13.207836 20770 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6fb3717db6bf4269ab7e3d2131cd415a: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 06:33:13.208527 20770 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6fb3717db6bf4269ab7e3d2131cd415a: Bootstrap complete.
I20260811 06:33:13.218541 20770 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6fb3717db6bf4269ab7e3d2131cd415a [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fb3717db6bf4269ab7e3d2131cd415a" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 33729 } }
I20260811 06:33:13.219501 20770 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6fb3717db6bf4269ab7e3d2131cd415a [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fb3717db6bf4269ab7e3d2131cd415a, State: Initialized, Role: FOLLOWER
I20260811 06:33:13.220122 20770 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6fb3717db6bf4269ab7e3d2131cd415a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fb3717db6bf4269ab7e3d2131cd415a" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 33729 } }
I20260811 06:33:13.220394 20770 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6fb3717db6bf4269ab7e3d2131cd415a [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:33:13.220531 20770 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6fb3717db6bf4269ab7e3d2131cd415a [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:33:13.220729 20770 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6fb3717db6bf4269ab7e3d2131cd415a [term 2 FOLLOWER]: Advancing to term 3
I20260811 06:33:13.224773 20770 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6fb3717db6bf4269ab7e3d2131cd415a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fb3717db6bf4269ab7e3d2131cd415a" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 33729 } }
I20260811 06:33:13.225354 20770 leader_election.cc:302] T 00000000000000000000000000000000 P 6fb3717db6bf4269ab7e3d2131cd415a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6fb3717db6bf4269ab7e3d2131cd415a; no voters:
I20260811 06:33:13.226122 20770 leader_election.cc:288] T 00000000000000000000000000000000 P 6fb3717db6bf4269ab7e3d2131cd415a [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 06:33:13.226271 20776 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6fb3717db6bf4269ab7e3d2131cd415a [term 3 FOLLOWER]: Leader election won for term 3
I20260811 06:33:13.227547 20776 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6fb3717db6bf4269ab7e3d2131cd415a [term 3 LEADER]: Becoming Leader. State: Replica: 6fb3717db6bf4269ab7e3d2131cd415a, State: Running, Role: LEADER
I20260811 06:33:13.228137 20776 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6fb3717db6bf4269ab7e3d2131cd415a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fb3717db6bf4269ab7e3d2131cd415a" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 33729 } }
I20260811 06:33:13.228550 20770 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6fb3717db6bf4269ab7e3d2131cd415a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:33:13.231097 20777 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6fb3717db6bf4269ab7e3d2131cd415a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "6fb3717db6bf4269ab7e3d2131cd415a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fb3717db6bf4269ab7e3d2131cd415a" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 33729 } } }
I20260811 06:33:13.231609 20777 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6fb3717db6bf4269ab7e3d2131cd415a [sys.catalog]: This master's current role is: LEADER
I20260811 06:33:13.232084 20778 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6fb3717db6bf4269ab7e3d2131cd415a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6fb3717db6bf4269ab7e3d2131cd415a. Latest consensus state: current_term: 3 leader_uuid: "6fb3717db6bf4269ab7e3d2131cd415a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fb3717db6bf4269ab7e3d2131cd415a" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 33729 } } }
I20260811 06:33:13.232396 20778 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6fb3717db6bf4269ab7e3d2131cd415a [sys.catalog]: This master's current role is: LEADER
I20260811 06:33:13.235562 20784 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:33:13.245116 20784 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:33:13.246677 20784 catalog_manager.cc:1259] Loaded cluster ID: 610c580947464c11955b0a775094e958
I20260811 06:33:13.246863 20784 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260811 06:33:13.250260 20784 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fb6145799ad google::LogMessage::Fail() at ??:0
@ 0x7fb61457c88a google::LogMessage::SendToLog() at ??:0
@ 0x7fb6145794de google::LogMessage::Flush() at ??:0
@ 0x7fb61457aa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fb629fb0512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7fb629fad976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fb62a035610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7fb62a035439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fb6171c5345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fb6171c891c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7fb6171c8799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fb6171a3eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7fb61a47d6db start_thread at ??:0
@ 0x7fb61341361f clone at ??:0
W20260811 06:33:13.592952 20773 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:33:13.593281 20773 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:33:13.593444 20773 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:33:13.600983 20773 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:33:13.601101 20773 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:33:13.601294 20773 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.193
I20260811 06:33:13.611847 20773 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:42797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.193.keytab
--principal=kudu/127.28.239.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.193:37147
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.239.193
--webserver_port=46097
--tserver_master_addrs=127.28.239.254:33729
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.193
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:33:13.613258 20773 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:33:13.614740 20773 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:33:13.628660 20827 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:33:13.628726 20826 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:33:13.628748 20829 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:33:13.630625 20773 server_base.cc:1034] running on GCE node
I20260811 06:33:13.631268 20773 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:33:13.632283 20773 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:33:13.633555 20773 hybrid_clock.cc:648] HybridClock initialized: now 1786429993633473 us; error 36 us; skew 500 ppm
I20260811 06:33:13.633991 20773 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:33:13 dist-test-slave-gkw7 krb5kdc[18901](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429993, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:33:13.637720 20773 init.cc:377] Logged in from keytab as kudu/127.28.239.193@KRBTEST.COM (short username kudu)
I20260811 06:33:13.639212 20773 webserver.cc:460] Webserver started at http://127.28.239.193:46097/ using document root <none> and password file <none>
I20260811 06:33:13.639973 20773 fs_manager.cc:359] Metadata directory not provided
I20260811 06:33:13.640110 20773 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:33:13.645785 20773 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260811 06:33:13.648679 20837 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:33:13.650128 20773 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:33:13.650324 20773 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-0/wal
uuid: "fc4e99a911a8498fb3fecd21d55491d9"
format_stamp: "Formatted at 2026-08-11 06:33:07 on dist-test-slave-gkw7"
server_key: "87229cf8f4d20f1ce8947f6f5b40d369"
server_key_iv: "1caf3cbbcbf5374eb3fe102526d0da06"
server_key_version: "encryptionkey@0"
I20260811 06:33:13.651368 20773 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:33:13.681459 20773 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:33:13.686292 20773 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:33:13.686616 20773 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:33:13.687693 20773 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:33:13.689143 20773 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:33:13.689234 20773 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:33:13.689395 20773 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:33:13.689502 20773 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:33:13.725674 20773 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.193:37147
I20260811 06:33:13.725708 21023 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.193:37147 every 8 connection(s)
I20260811 06:33:13.727262 20773 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
I20260811 06:33:13.731796 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 20773
I20260811 06:33:13.732622 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.194:37153
--local_ip_for_outbound_sockets=127.28.239.194
--tserver_master_addrs=127.28.239.254:33729
--webserver_port=44943
--webserver_interface=127.28.239.194
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.194.keytab
--principal=kudu/127.28.239.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:42797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 06:33:13.733057 21026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:13.730499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:33729 (local address 127.28.239.193:39279)
0811 06:33:13.731389 (+ 890us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:13.731717 (+ 328us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":283,"threads_started":1}
W20260811 06:33:13.733866 21024 heartbeater.cc:644] Failed to heartbeat to 127.28.239.254:33729 (0 consecutive failures): Network error: Failed to ping master at 127.28.239.254:33729: Client connection negotiation failed: client connection to 127.28.239.254:33729: connect: Connection refused (error 111)
I20260811 06:33:13.734947 21026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:13.734457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:33729 (local address 127.28.239.193:44485)
0811 06:33:13.734670 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:13.734779 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260811 06:33:13.735961 21026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:13.735530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:33729 (local address 127.28.239.193:48817)
0811 06:33:13.735699 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:13.735803 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260811 06:33:13.736253 21024 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 06:33:14.081460 21029 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:33:14.081864 21029 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:33:14.082068 21029 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:33:14.089419 21029 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:33:14.089584 21029 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:33:14.089797 21029 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.194
I20260811 06:33:14.099802 21029 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:42797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.194.keytab
--principal=kudu/127.28.239.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.194:37153
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.239.194
--webserver_port=44943
--tserver_master_addrs=127.28.239.254:33729
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.194
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:33:14.101166 21029 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:33:14.102617 21029 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:33:14.119897 21038 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:33:14.119897 21040 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:33:14.120615 21043 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:33:14.121160 21029 server_base.cc:1034] running on GCE node
I20260811 06:33:14.121987 21029 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:33:14.123189 21029 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:33:14.124439 21029 hybrid_clock.cc:648] HybridClock initialized: now 1786429994124363 us; error 67 us; skew 500 ppm
I20260811 06:33:14.124825 21029 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:33:14 dist-test-slave-gkw7 krb5kdc[18901](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429994, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:33:14.128512 21029 init.cc:377] Logged in from keytab as kudu/127.28.239.194@KRBTEST.COM (short username kudu)
I20260811 06:33:14.130414 21029 webserver.cc:460] Webserver started at http://127.28.239.194:44943/ using document root <none> and password file <none>
I20260811 06:33:14.131183 21029 fs_manager.cc:359] Metadata directory not provided
I20260811 06:33:14.131359 21029 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:33:14.137372 21029 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 06:33:14.140529 21057 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:33:14.142340 21029 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 06:33:14.142529 21029 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-1/wal
uuid: "91ec4d92310b4496a89c1eecf66102a3"
format_stamp: "Formatted at 2026-08-11 06:33:07 on dist-test-slave-gkw7"
server_key: "37f6f06d05194a49812e2141c9e9a675"
server_key_iv: "8b0ff52b36eeddcaff3c89b29eb22656"
server_key_version: "encryptionkey@0"
I20260811 06:33:14.143591 21029 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:33:14.167794 21029 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:33:14.172816 21029 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:33:14.173125 21029 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:33:14.174381 21029 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:33:14.175896 21029 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:33:14.176043 21029 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:33:14.176183 21029 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:33:14.176275 21029 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:33:14.215659 21029 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.194:37153
I20260811 06:33:14.215704 21233 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.194:37153 every 8 connection(s)
I20260811 06:33:14.217231 21029 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
I20260811 06:33:14.223448 21236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:14.220565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:33729 (local address 127.28.239.194:55585)
0811 06:33:14.221551 (+ 986us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:14.221929 (+ 378us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":583,"thread_start_us":336,"threads_started":1}
I20260811 06:33:14.224215 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 21029
W20260811 06:33:14.224349 21234 heartbeater.cc:644] Failed to heartbeat to 127.28.239.254:33729 (0 consecutive failures): Network error: Failed to ping master at 127.28.239.254:33729: Client connection negotiation failed: client connection to 127.28.239.254:33729: connect: Connection refused (error 111)
I20260811 06:33:14.224961 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.195:36329
--local_ip_for_outbound_sockets=127.28.239.195
--tserver_master_addrs=127.28.239.254:33729
--webserver_port=33353
--webserver_interface=127.28.239.195
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.195.keytab
--principal=kudu/127.28.239.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:42797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 06:33:14.225687 21236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:14.224977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:33729 (local address 127.28.239.194:39607)
0811 06:33:14.225377 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:14.225480 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":261}
I20260811 06:33:14.226786 21236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:14.226305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:33729 (local address 127.28.239.194:33213)
0811 06:33:14.226500 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:14.226609 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260811 06:33:14.227154 21234 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 06:33:14.596652 21237 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:33:14.596935 21237 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:33:14.597103 21237 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:33:14.604396 21237 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:33:14.604511 21237 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:33:14.604743 21237 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.195
I20260811 06:33:14.614717 21237 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:42797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.195.keytab
--principal=kudu/127.28.239.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.195:36329
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.239.195
--webserver_port=33353
--tserver_master_addrs=127.28.239.254:33729
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.195
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:33:14.616145 21237 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:33:14.617592 21237 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:33:14.631757 21256 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:33:14.631812 21258 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:33:14.631759 21252 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:33:14.632573 21237 server_base.cc:1034] running on GCE node
I20260811 06:33:14.633252 21237 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:33:14.634339 21237 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:33:14.635591 21237 hybrid_clock.cc:648] HybridClock initialized: now 1786429994635537 us; error 40 us; skew 500 ppm
I20260811 06:33:14.635933 21237 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:33:14 dist-test-slave-gkw7 krb5kdc[18901](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786429994, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:33:14.645200 21237 init.cc:377] Logged in from keytab as kudu/127.28.239.195@KRBTEST.COM (short username kudu)
I20260811 06:33:14.647104 21237 webserver.cc:460] Webserver started at http://127.28.239.195:33353/ using document root <none> and password file <none>
I20260811 06:33:14.647902 21237 fs_manager.cc:359] Metadata directory not provided
I20260811 06:33:14.648093 21237 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:33:14.654448 21237 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 06:33:14.657387 21270 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:33:14.658890 21237 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 06:33:14.659106 21237 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-2/wal
uuid: "bb42fe5d26884700bb28ac7c1da33596"
format_stamp: "Formatted at 2026-08-11 06:33:08 on dist-test-slave-gkw7"
server_key: "984b476d0ce85290bf9011a0ea33b0b3"
server_key_iv: "e038ec5b7d17787c35cc14ff98ca44b3"
server_key_version: "encryptionkey@0"
I20260811 06:33:14.660156 21237 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:33:14.681311 21237 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:33:14.685490 21237 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:33:14.685847 21237 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:33:14.686897 21237 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:33:14.688301 21237 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:33:14.688424 21237 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:33:14.688570 21237 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:33:14.688661 21237 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:33:14.723592 21237 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.195:36329
I20260811 06:33:14.723685 21445 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.195:36329 every 8 connection(s)
I20260811 06:33:14.725226 21237 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
I20260811 06:33:14.727633 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 21237
I20260811 06:33:14.731094 21450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:14.728833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:33729 (local address 127.28.239.195:49469)
0811 06:33:14.729943 (+ 1110us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:14.730213 (+ 270us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":614,"thread_start_us":380,"threads_started":1}
W20260811 06:33:14.731736 21447 heartbeater.cc:644] Failed to heartbeat to 127.28.239.254:33729 (0 consecutive failures): Network error: Failed to ping master at 127.28.239.254:33729: Client connection negotiation failed: client connection to 127.28.239.254:33729: connect: Connection refused (error 111)
I20260811 06:33:14.732569 21450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:14.732239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:33729 (local address 127.28.239.195:55459)
0811 06:33:14.732396 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:14.732472 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260811 06:33:14.733412 21450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:14.733081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:33729 (local address 127.28.239.195:59337)
0811 06:33:14.733219 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:14.733304 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260811 06:33:14.733662 21447 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 06:33:14.737363 21451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:14.736708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:33729 (local address 127.28.239.193:38215)
0811 06:33:14.737084 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:14.737186 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":186,"threads_started":1}
I20260811 06:33:15.228832 21452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:15.227842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:33729 (local address 127.28.239.194:46421)
0811 06:33:15.228449 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:15.228578 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":333,"threads_started":1}
I20260811 06:33:15.735123 21453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:15.734226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:33729 (local address 127.28.239.195:52365)
0811 06:33:15.734751 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:15.734892 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":246,"threads_started":1}
I20260811 06:33:15.739063 21454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:15.738308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:33729 (local address 127.28.239.193:37245)
0811 06:33:15.738812 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:15.738911 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":240,"threads_started":1}
I20260811 06:33:16.230662 21455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:16.229868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:33729 (local address 127.28.239.194:33283)
0811 06:33:16.230410 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:16.230492 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":282,"threads_started":1}
I20260811 06:33:16.737068 21457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:16.736151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:33729 (local address 127.28.239.195:43187)
0811 06:33:16.736678 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:16.736765 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":284,"threads_started":1}
I20260811 06:33:16.740623 21459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:16.739916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:33729 (local address 127.28.239.193:56431)
0811 06:33:16.740362 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:16.740444 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":227,"threads_started":1}
I20260811 06:33:17.232364 21460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:17.231637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:33729 (local address 127.28.239.194:37925)
0811 06:33:17.232096 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:17.232178 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":234,"threads_started":1}
I20260811 06:33:17.738950 21462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:17.738159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:33729 (local address 127.28.239.195:35201)
0811 06:33:17.738664 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:17.738773 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":287,"threads_started":1}
I20260811 06:33:17.742377 21463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:17.741681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:33729 (local address 127.28.239.193:51471)
0811 06:33:17.742113 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:17.742197 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":225,"threads_started":1}
I20260811 06:33:18.234330 21464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:18.233329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:33729 (local address 127.28.239.194:43227)
0811 06:33:18.233941 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:18.234066 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":326,"threads_started":1}
I20260811 06:33:18.740954 21469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:18.740100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:33729 (local address 127.28.239.195:40195)
0811 06:33:18.740664 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:18.740775 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":282,"threads_started":1}
I20260811 06:33:18.744017 21471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:18.743336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:33729 (local address 127.28.239.193:44395)
0811 06:33:18.743761 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:18.743841 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":222,"threads_started":1}
I20260811 06:33:19.236207 21475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:19.235363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:33729 (local address 127.28.239.194:52009)
0811 06:33:19.235884 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:19.235969 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":272,"threads_started":1}
I20260811 06:33:19.742686 21476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:19.741950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:33729 (local address 127.28.239.195:33239)
0811 06:33:19.742421 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:19.742513 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":236,"threads_started":1}
I20260811 06:33:19.745740 21479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:19.744965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:33729 (local address 127.28.239.193:35881)
0811 06:33:19.745403 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:19.745494 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":228,"threads_started":1}
I20260811 06:33:20.238225 21481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:20.237321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:33729 (local address 127.28.239.194:60537)
0811 06:33:20.237919 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:20.238023 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":361,"threads_started":1}
I20260811 06:33:20.744355 21483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:20.743628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:33729 (local address 127.28.239.195:42815)
0811 06:33:20.744078 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:20.744171 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":227,"threads_started":1}
I20260811 06:33:20.747509 21485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:20.746696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:33729 (local address 127.28.239.193:57021)
0811 06:33:20.747202 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:20.747299 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":209,"threads_started":1}
I20260811 06:33:21.240202 21486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:21.239302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:33729 (local address 127.28.239.194:52525)
0811 06:33:21.239878 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:21.239968 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":255,"threads_started":1}
I20260811 06:33:21.746269 21489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:21.745325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:33729 (local address 127.28.239.195:60129)
0811 06:33:21.745925 (+ 600us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:21.746020 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":299,"threads_started":1}
I20260811 06:33:21.749336 21490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:21.748592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:33729 (local address 127.28.239.193:58967)
0811 06:33:21.749085 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:21.749179 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":271,"threads_started":1}
I20260811 06:33:22.242204 21491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:22.241366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:33729 (local address 127.28.239.194:37371)
0811 06:33:22.241943 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:22.242029 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":330,"threads_started":1}
I20260811 06:33:22.748098 21492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:22.747316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:33729 (local address 127.28.239.195:40571)
0811 06:33:22.747802 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:22.747882 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":289,"threads_started":1}
I20260811 06:33:22.751322 21493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:22.750459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:33729 (local address 127.28.239.193:52475)
0811 06:33:22.751027 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:22.751138 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":325,"threads_started":1}
I20260811 06:33:23.244072 21494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:23.243233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:33729 (local address 127.28.239.194:36411)
0811 06:33:23.243792 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:23.243884 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":250,"threads_started":1}
I20260811 06:33:23.749945 21495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:23.749025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:33729 (local address 127.28.239.195:33161)
0811 06:33:23.749662 (+ 637us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:23.749767 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":236,"threads_started":1}
I20260811 06:33:23.753136 21497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:23.752332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:33729 (local address 127.28.239.193:48199)
0811 06:33:23.752829 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:23.752962 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":294,"threads_started":1}
I20260811 06:33:24.245971 21499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:24.245164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:33729 (local address 127.28.239.194:59097)
0811 06:33:24.245714 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:24.245802 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":256,"threads_started":1}
I20260811 06:33:24.751593 21500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:24.750884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:33729 (local address 127.28.239.195:43839)
0811 06:33:24.751336 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:24.751430 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":235,"threads_started":1}
I20260811 06:33:24.754914 21502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:24.754119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:33729 (local address 127.28.239.193:33121)
0811 06:33:24.754591 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:24.754672 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":234,"threads_started":1}
I20260811 06:33:25.247818 21506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:25.247054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:33729 (local address 127.28.239.194:33329)
0811 06:33:25.247527 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:25.247634 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":249,"threads_started":1}
I20260811 06:33:25.753403 21507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:25.752637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:33729 (local address 127.28.239.195:41993)
0811 06:33:25.753108 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:25.753202 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":243,"threads_started":1}
I20260811 06:33:25.756788 21509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:25.755834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:33729 (local address 127.28.239.193:37749)
0811 06:33:25.756361 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:25.756518 (+ 157us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":263,"threads_started":1}
I20260811 06:33:26.249626 21510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:26.248791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:33729 (local address 127.28.239.194:50089)
0811 06:33:26.249305 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:26.249405 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":283,"threads_started":1}
I20260811 06:33:26.755295 21512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:26.754449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:33729 (local address 127.28.239.195:45531)
0811 06:33:26.755027 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:26.755120 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":287,"threads_started":1}
I20260811 06:33:26.758507 21514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:26.757773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:33729 (local address 127.28.239.193:58443)
0811 06:33:26.758205 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:26.758331 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":217,"threads_started":1}
I20260811 06:33:27.251385 21517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:27.250557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:33729 (local address 127.28.239.194:57933)
0811 06:33:27.251078 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:27.251184 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":259,"threads_started":1}
I20260811 06:33:27.757218 21521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:27.756371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:33729 (local address 127.28.239.195:48051)
0811 06:33:27.756933 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:27.757029 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":261,"threads_started":1}
I20260811 06:33:27.760206 21523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:27.759537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:33729 (local address 127.28.239.193:33473)
0811 06:33:27.759953 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:27.760035 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":216,"threads_started":1}
I20260811 06:33:28.253082 21524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:28.252319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:33729 (local address 127.28.239.194:49485)
0811 06:33:28.252801 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:28.252889 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":245,"threads_started":1}
I20260811 06:33:28.759016 21525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:28.758189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:33729 (local address 127.28.239.195:41889)
0811 06:33:28.758728 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:28.758831 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":297,"threads_started":1}
I20260811 06:33:28.762422 21529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:28.761465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:33729 (local address 127.28.239.193:43095)
0811 06:33:28.762134 (+ 669us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:28.762243 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":527,"thread_start_us":385,"threads_started":1}
I20260811 06:33:29.255376 21530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:29.254421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:33729 (local address 127.28.239.194:41391)
0811 06:33:29.255017 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:29.255160 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":297,"threads_started":1}
I20260811 06:33:29.731487 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 20773
I20260811 06:33:29.761317 21532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:29.760304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:33729 (local address 127.28.239.195:54229)
0811 06:33:29.760904 (+ 600us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:29.761041 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":257,"threads_started":1}
I20260811 06:33:29.761377 21531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:29.732699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46246 (local address 127.28.239.193:37147)
0811 06:33:29.733412 (+ 713us) server_negotiation.cc:206] Beginning negotiation
0811 06:33:29.733431 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 06:33:29.733477 (+ 46us) server_negotiation.cc:407] Connection header received
0811 06:33:29.735782 (+ 2305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:33:29.735793 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:33:29.736275 (+ 482us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:33:29.736665 (+ 390us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:33:29.738844 (+ 2179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:29.740206 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:29.741743 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:29.742532 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:29.742717 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:29.751662 (+ 8945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:33:29.751673 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:33:29.751693 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:33:29.751744 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:33:29.755914 (+ 4170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:33:29.757116 (+ 1202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:33:29.757125 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:33:29.757133 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:33:29.757325 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:33:29.757860 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:33:29.757869 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:33:29.757875 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:33:29.758788 (+ 913us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:33:29.759146 (+ 358us) server_negotiation.cc:1035] Waiting for connection context
0811 06:33:29.760295 (+ 1149us) server_negotiation.cc:299] Negotiation successful
0811 06:33:29.760895 (+ 600us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":505,"thread_start_us":319,"threads_started":1}
I20260811 06:33:29.763394 21016 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:33:29.850373 21533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:29.763316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:33729 (local address 127.28.239.193:52383)
0811 06:33:29.849786 (+ 86470us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:29.850005 (+ 219us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:33729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86323,"thread_start_us":86083,"threads_started":1}
I20260811 06:33:29.851140 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 20773
I20260811 06:33:29.881138 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 21029
I20260811 06:33:29.899856 21536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:29.882261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33810 (local address 127.28.239.194:37153)
0811 06:33:29.882920 (+ 659us) server_negotiation.cc:206] Beginning negotiation
0811 06:33:29.882935 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 06:33:29.882976 (+ 41us) server_negotiation.cc:407] Connection header received
0811 06:33:29.884557 (+ 1581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:33:29.884568 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:33:29.885007 (+ 439us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:33:29.885358 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:33:29.887000 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:29.888422 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:29.889785 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:29.890483 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:29.890629 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:29.893368 (+ 2739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:33:29.893376 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:33:29.893395 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:33:29.893442 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:33:29.896610 (+ 3168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:33:29.897324 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:33:29.897331 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:33:29.897339 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:33:29.897518 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:33:29.897946 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:33:29.897949 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:33:29.897951 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:33:29.898525 (+ 574us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:33:29.898767 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0811 06:33:29.899202 (+ 435us) server_negotiation.cc:299] Negotiation successful
0811 06:33:29.899561 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":485,"thread_start_us":326,"threads_started":1}
I20260811 06:33:29.901409 21228 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:33:29.986763 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 21029
I20260811 06:33:30.010265 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 21237
I20260811 06:33:30.028820 21538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:30.011547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44624 (local address 127.28.239.195:36329)
0811 06:33:30.012289 (+ 742us) server_negotiation.cc:206] Beginning negotiation
0811 06:33:30.012304 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 06:33:30.012349 (+ 45us) server_negotiation.cc:407] Connection header received
0811 06:33:30.014112 (+ 1763us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:33:30.014122 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:33:30.014601 (+ 479us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:33:30.014905 (+ 304us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:33:30.016274 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:30.017491 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:30.018875 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:30.019541 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:30.019686 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:30.022605 (+ 2919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:33:30.022614 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:33:30.022630 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:33:30.022671 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:33:30.025604 (+ 2933us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:33:30.026358 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:33:30.026365 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:33:30.026372 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:33:30.026544 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:33:30.026884 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:33:30.026888 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:33:30.026890 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:33:30.027443 (+ 553us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:33:30.027665 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0811 06:33:30.028021 (+ 356us) server_negotiation.cc:299] Negotiation successful
0811 06:33:30.028514 (+ 493us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":543,"thread_start_us":370,"threads_started":1}
I20260811 06:33:30.030382 21441 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:33:30.115314 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 21237
2026-08-11T06:33:30Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (25532 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:33:30 dist-test-slave-gkw7 krb5kdc[21544](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:33:30 dist-test-slave-gkw7 krb5kdc[21544](info): set up 2 sockets
Aug 11 06:33:30 dist-test-slave-gkw7 krb5kdc[21544](info): commencing operation
krb5kdc: starting...
W20260811 06:33:31.570240 29631 mini_kdc.cc:121] Time spent starting KDC: real 1.406s user 0.003s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/krb5kdc/test-user.keytab.
Aug 11 06:33:31 dist-test-slave-gkw7 krb5kdc[21544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430011, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:33:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:33:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.239.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.239.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.239.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab.
Entry for principal kudu/127.28.239.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab.
I20260811 06:33:31.889416 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:42189
--webserver_interface=127.28.239.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab
--principal=kudu/127.28.239.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:43401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.239.254:42189
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:33:32.249058 21565 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:33:32.249447 21565 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:33:32.249642 21565 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:33:32.257133 21565 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 06:33:32.257284 21565 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 06:33:32.257381 21565 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 06:33:32.257490 21565 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:33:32.257611 21565 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:33:32.257742 21565 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:33:32.257833 21565 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:33:32.257913 21565 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:33:32.268500 21565 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:43401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.28.239.254:42189
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab
--principal=kudu/127.28.239.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:42189
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.239.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:33:32.270011 21565 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:33:32.271423 21565 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:33:32.284572 21572 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:33:32.284466 21577 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:33:32.284760 21565 server_base.cc:1034] running on GCE node
W20260811 06:33:32.284520 21574 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:33:32.286075 21565 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:33:32.287624 21565 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:33:32.288849 21565 hybrid_clock.cc:648] HybridClock initialized: now 1786430012288766 us; error 57 us; skew 500 ppm
I20260811 06:33:32.289216 21565 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:33:32 dist-test-slave-gkw7 krb5kdc[21544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430012, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:33:32.292783 21565 init.cc:377] Logged in from keytab as kudu/127.28.239.254@KRBTEST.COM (short username kudu)
I20260811 06:33:32.294344 21565 webserver.cc:460] Webserver started at http://127.28.239.254:37769/ using document root <none> and password file <none>
I20260811 06:33:32.295107 21565 fs_manager.cc:359] Metadata directory not provided
I20260811 06:33:32.295207 21565 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:33:32.295588 21565 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:33:32.298805 21565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/master-0/data/instance:
uuid: "f9315cc436a5414ea115e916586d7497"
format_stamp: "Formatted at 2026-08-11 06:33:32 on dist-test-slave-gkw7"
server_key: "7f5f786b8c78515694591e9397b4c3e3"
server_key_iv: "198e4780229f0fb00f84ac5a2090c859"
server_key_version: "encryptionkey@0"
I20260811 06:33:32.299674 21565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/master-0/wal/instance:
uuid: "f9315cc436a5414ea115e916586d7497"
format_stamp: "Formatted at 2026-08-11 06:33:32 on dist-test-slave-gkw7"
server_key: "7f5f786b8c78515694591e9397b4c3e3"
server_key_iv: "198e4780229f0fb00f84ac5a2090c859"
server_key_version: "encryptionkey@0"
I20260811 06:33:32.305359 21565 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260811 06:33:32.309048 21586 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:33:32.310730 21565 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:33:32.310981 21565 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/master-0/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/master-0/wal
uuid: "f9315cc436a5414ea115e916586d7497"
format_stamp: "Formatted at 2026-08-11 06:33:32 on dist-test-slave-gkw7"
server_key: "7f5f786b8c78515694591e9397b4c3e3"
server_key_iv: "198e4780229f0fb00f84ac5a2090c859"
server_key_version: "encryptionkey@0"
I20260811 06:33:32.311306 21565 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:33:32.328967 21565 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:33:32.333494 21565 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:33:32.333796 21565 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:33:32.354554 21565 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.254:42189
I20260811 06:33:32.354591 21667 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.254:42189 every 8 connection(s)
I20260811 06:33:32.356137 21565 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
I20260811 06:33:32.360034 21668 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:33:32.361603 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 21565
I20260811 06:33:32.361874 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/master-0/wal/instance
I20260811 06:33:32.362542 29631 external_mini_cluster.cc:1428] Setting key 55755241a6527b7cbe7334b9bd9ee9c9
I20260811 06:33:32.372742 21668 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9315cc436a5414ea115e916586d7497: Bootstrap starting.
I20260811 06:33:32.376901 21668 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f9315cc436a5414ea115e916586d7497: Neither blocks nor log segments found. Creating new log.
Aug 11 06:33:32 dist-test-slave-gkw7 krb5kdc[21544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430011, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:33:32.378227 21668 log.cc:824] T 00000000000000000000000000000000 P f9315cc436a5414ea115e916586d7497: Log is configured to *not* fsync() on all Append() calls
I20260811 06:33:32.382079 21668 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9315cc436a5414ea115e916586d7497: No bootstrap required, opened a new log
I20260811 06:33:32.389390 21672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:32.364781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41248 (local address 127.28.239.254:42189)
0811 06:33:32.365712 (+ 931us) server_negotiation.cc:206] Beginning negotiation
0811 06:33:32.365725 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 06:33:32.365773 (+ 48us) server_negotiation.cc:407] Connection header received
0811 06:33:32.367484 (+ 1711us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:33:32.367507 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:33:32.367910 (+ 403us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:33:32.368236 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:33:32.370201 (+ 1965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:32.371573 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:32.373465 (+ 1892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:32.374185 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:32.374321 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:32.379774 (+ 5453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:33:32.379788 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:33:32.379812 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:33:32.379867 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:33:32.384038 (+ 4171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:33:32.384970 (+ 932us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:33:32.384986 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:33:32.385000 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:33:32.385230 (+ 230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:33:32.385819 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:33:32.385824 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:33:32.385827 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:33:32.386630 (+ 803us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:33:32.386933 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0811 06:33:32.387384 (+ 451us) server_negotiation.cc:299] Negotiation successful
0811 06:33:32.387900 (+ 516us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":667,"thread_start_us":422,"threads_started":1}
I20260811 06:33:32.392858 21668 raft_consensus.cc:348] T 00000000000000000000000000000000 P f9315cc436a5414ea115e916586d7497 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9315cc436a5414ea115e916586d7497" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 42189 } }
I20260811 06:33:32.393193 21668 raft_consensus.cc:374] T 00000000000000000000000000000000 P f9315cc436a5414ea115e916586d7497 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:33:32.393270 21668 raft_consensus.cc:729] T 00000000000000000000000000000000 P f9315cc436a5414ea115e916586d7497 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9315cc436a5414ea115e916586d7497, State: Initialized, Role: FOLLOWER
I20260811 06:33:32.393865 21668 consensus_queue.cc:260] T 00000000000000000000000000000000 P f9315cc436a5414ea115e916586d7497 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9315cc436a5414ea115e916586d7497" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 42189 } }
I20260811 06:33:32.394071 21668 raft_consensus.cc:388] T 00000000000000000000000000000000 P f9315cc436a5414ea115e916586d7497 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:33:32.394172 21668 raft_consensus.cc:482] T 00000000000000000000000000000000 P f9315cc436a5414ea115e916586d7497 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:33:32.394299 21668 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f9315cc436a5414ea115e916586d7497 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:33:32.396466 21668 raft_consensus.cc:504] T 00000000000000000000000000000000 P f9315cc436a5414ea115e916586d7497 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9315cc436a5414ea115e916586d7497" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 42189 } }
I20260811 06:33:32.396940 21668 leader_election.cc:302] T 00000000000000000000000000000000 P f9315cc436a5414ea115e916586d7497 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f9315cc436a5414ea115e916586d7497; no voters:
I20260811 06:33:32.397675 21668 leader_election.cc:288] T 00000000000000000000000000000000 P f9315cc436a5414ea115e916586d7497 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:33:32.397856 21675 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f9315cc436a5414ea115e916586d7497 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:33:32.398798 21675 raft_consensus.cc:686] T 00000000000000000000000000000000 P f9315cc436a5414ea115e916586d7497 [term 1 LEADER]: Becoming Leader. State: Replica: f9315cc436a5414ea115e916586d7497, State: Running, Role: LEADER
I20260811 06:33:32.399416 21675 consensus_queue.cc:237] T 00000000000000000000000000000000 P f9315cc436a5414ea115e916586d7497 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9315cc436a5414ea115e916586d7497" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 42189 } }
I20260811 06:33:32.399971 21668 sys_catalog.cc:564] T 00000000000000000000000000000000 P f9315cc436a5414ea115e916586d7497 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:33:32.402508 21677 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9315cc436a5414ea115e916586d7497 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f9315cc436a5414ea115e916586d7497. Latest consensus state: current_term: 1 leader_uuid: "f9315cc436a5414ea115e916586d7497" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9315cc436a5414ea115e916586d7497" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 42189 } } }
I20260811 06:33:32.402818 21676 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9315cc436a5414ea115e916586d7497 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f9315cc436a5414ea115e916586d7497" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9315cc436a5414ea115e916586d7497" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 42189 } } }
I20260811 06:33:32.403463 21676 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9315cc436a5414ea115e916586d7497 [sys.catalog]: This master's current role is: LEADER
I20260811 06:33:32.403493 21677 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9315cc436a5414ea115e916586d7497 [sys.catalog]: This master's current role is: LEADER
I20260811 06:33:32.405979 21683 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:33:32.411558 21683 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:33:32.421108 21683 catalog_manager.cc:1347] Generated new cluster ID: 8ea31dfa1ea24bde8f419dac748e64f0
I20260811 06:33:32.421232 21683 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:33:32.432132 21683 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:33:32.433477 21683 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:33:32.440979 21683 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f9315cc436a5414ea115e916586d7497: Generated new TSK 0
I20260811 06:33:32.441926 21683 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.239.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.239.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.239.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.193.keytab.
Entry for principal kudu/127.28.239.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.193.keytab.
I20260811 06:33:32.515805 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.193:0
--local_ip_for_outbound_sockets=127.28.239.193
--webserver_interface=127.28.239.193
--webserver_port=0
--tserver_master_addrs=127.28.239.254:42189
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.193.keytab
--principal=kudu/127.28.239.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:43401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:33:32.871789 21711 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:33:32.872087 21711 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:33:32.872188 21711 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:33:32.879804 21711 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:33:32.879928 21711 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:33:32.880106 21711 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.193
I20260811 06:33:32.890916 21711 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:43401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.193.keytab
--principal=kudu/127.28.239.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.239.193
--webserver_port=0
--tserver_master_addrs=127.28.239.254:42189
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.193
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:33:32.892434 21711 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:33:32.893980 21711 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:33:32.908478 21721 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:33:32.908689 21717 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:33:32.908425 21718 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:33:32.909363 21711 server_base.cc:1034] running on GCE node
I20260811 06:33:32.910035 21711 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:33:32.911005 21711 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:33:32.912215 21711 hybrid_clock.cc:648] HybridClock initialized: now 1786430012912157 us; error 46 us; skew 500 ppm
I20260811 06:33:32.912510 21711 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:33:32 dist-test-slave-gkw7 krb5kdc[21544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430012, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:33:32.915920 21711 init.cc:377] Logged in from keytab as kudu/127.28.239.193@KRBTEST.COM (short username kudu)
I20260811 06:33:32.917497 21711 webserver.cc:460] Webserver started at http://127.28.239.193:42879/ using document root <none> and password file <none>
I20260811 06:33:32.918311 21711 fs_manager.cc:359] Metadata directory not provided
I20260811 06:33:32.918455 21711 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:33:32.918799 21711 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:33:32.922034 21711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-0/data/instance:
uuid: "5e51526ceea347ba88b10bc1a50d84ca"
format_stamp: "Formatted at 2026-08-11 06:33:32 on dist-test-slave-gkw7"
server_key: "5cf5d515f041520e8b8894e158130968"
server_key_iv: "9bf9a52c50f3d66cdf0d1c57c1f9e85f"
server_key_version: "encryptionkey@0"
I20260811 06:33:32.922856 21711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-0/wal/instance:
uuid: "5e51526ceea347ba88b10bc1a50d84ca"
format_stamp: "Formatted at 2026-08-11 06:33:32 on dist-test-slave-gkw7"
server_key: "5cf5d515f041520e8b8894e158130968"
server_key_iv: "9bf9a52c50f3d66cdf0d1c57c1f9e85f"
server_key_version: "encryptionkey@0"
I20260811 06:33:32.928318 21711 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 06:33:32.932108 21730 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:33:32.933518 21711 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 06:33:32.933763 21711 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-0/wal
uuid: "5e51526ceea347ba88b10bc1a50d84ca"
format_stamp: "Formatted at 2026-08-11 06:33:32 on dist-test-slave-gkw7"
server_key: "5cf5d515f041520e8b8894e158130968"
server_key_iv: "9bf9a52c50f3d66cdf0d1c57c1f9e85f"
server_key_version: "encryptionkey@0"
I20260811 06:33:32.933984 21711 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:33:32.972580 21711 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:33:32.977684 21711 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:33:32.978032 21711 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:33:32.979138 21711 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:33:32.980683 21711 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:33:32.980805 21711 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:33:32.980970 21711 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:33:32.981047 21711 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:33:33.018420 21711 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.193:42485
I20260811 06:33:33.018520 21893 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.193:42485 every 8 connection(s)
I20260811 06:33:33.019862 21711 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
I20260811 06:33:33.027942 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 21711
I20260811 06:33:33.028184 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-0/wal/instance
I20260811 06:33:33.028892 29631 external_mini_cluster.cc:1428] Setting key 76dfff3fda6b7824a1a2becb72392342
Aug 11 06:33:33 dist-test-slave-gkw7 krb5kdc[21544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430012, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.193@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:33:33.044112 21896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:33.023302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.193:50841 (local address 127.28.239.254:42189)
0811 06:33:33.023811 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0811 06:33:33.023816 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:33:33.024970 (+ 1154us) server_negotiation.cc:407] Connection header received
0811 06:33:33.027086 (+ 2116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:33:33.027094 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:33:33.027214 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:33:33.027346 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:33:33.029214 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:33.030216 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:33.031886 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:33.032336 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:33.032490 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:33.036462 (+ 3972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:33:33.036467 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:33:33.036470 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:33:33.036504 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:33:33.039713 (+ 3209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:33:33.040860 (+ 1147us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:33:33.040871 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:33:33.040875 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:33:33.041082 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:33:33.041709 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:33:33.041724 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:33:33.041728 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:33:33.042169 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:33:33.042380 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0811 06:33:33.043280 (+ 900us) server_negotiation.cc:299] Negotiation successful
0811 06:33:33.043592 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":253,"threads_started":1}
I20260811 06:33:33.045275 21898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:33.023918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:42189 (local address 127.28.239.193:50841)
0811 06:33:33.024685 (+ 767us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:33.024736 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 06:33:33.026774 (+ 2038us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:33:33.027626 (+ 852us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:33:33.027638 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:33:33.028340 (+ 702us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:33:33.029004 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:33:33.029039 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:33:33.030414 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:33:33.030418 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:33:33.031728 (+ 1310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:33:33.031741 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:33:33.032586 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:33:33.032595 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:33:33.032860 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:33.033670 (+ 810us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:33:33.033696 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:33:33.036223 (+ 2527us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:33:33.039898 (+ 3675us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:33:33.039909 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:33:33.039932 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:33:33.040549 (+ 617us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:33:33.041237 (+ 688us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:33:33.041244 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:33:33.041249 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:33:33.041501 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:33:33.042358 (+ 857us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:33:33.042368 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:33:33.042808 (+ 440us) client_negotiation.cc:769] Sending connection context
0811 06:33:33.043144 (+ 336us) client_negotiation.cc:240] Negotiation successful
0811 06:33:33.043823 (+ 679us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":315,"threads_started":1}
I20260811 06:33:33.047303 21894 heartbeater.cc:347] Connected to a master server at 127.28.239.254:42189
I20260811 06:33:33.047729 21894 heartbeater.cc:464] Registering TS with master...
I20260811 06:33:33.048676 21894 heartbeater.cc:511] Master 127.28.239.254:42189 requested a full tablet report, sending...
I20260811 06:33:33.051479 21613 ts_manager.cc:194] Registered new tserver with Master: 5e51526ceea347ba88b10bc1a50d84ca (127.28.239.193:42485)
I20260811 06:33:33.053925 21613 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.239.193@KRBTEST.COM'} at 127.28.239.193:50841
WARNING: no policy specified for kudu/127.28.239.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.239.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.239.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.194.keytab.
Entry for principal kudu/127.28.239.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.194.keytab.
I20260811 06:33:33.102885 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.194:0
--local_ip_for_outbound_sockets=127.28.239.194
--webserver_interface=127.28.239.194
--webserver_port=0
--tserver_master_addrs=127.28.239.254:42189
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.194.keytab
--principal=kudu/127.28.239.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:43401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:33:33.470311 21903 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:33:33.470620 21903 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:33:33.470721 21903 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:33:33.478065 21903 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:33:33.478181 21903 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:33:33.478294 21903 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.194
I20260811 06:33:33.488523 21903 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:43401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.194.keytab
--principal=kudu/127.28.239.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.239.194
--webserver_port=0
--tserver_master_addrs=127.28.239.254:42189
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.194
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:33:33.489936 21903 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:33:33.491350 21903 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:33:33.505869 21909 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:33:33.505810 21911 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:33:33.505781 21908 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:33:33.507426 21903 server_base.cc:1034] running on GCE node
I20260811 06:33:33.508100 21903 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:33:33.509203 21903 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:33:33.510445 21903 hybrid_clock.cc:648] HybridClock initialized: now 1786430013510369 us; error 77 us; skew 500 ppm
I20260811 06:33:33.510805 21903 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:33:33 dist-test-slave-gkw7 krb5kdc[21544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430013, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:33:33.514648 21903 init.cc:377] Logged in from keytab as kudu/127.28.239.194@KRBTEST.COM (short username kudu)
I20260811 06:33:33.516878 21903 webserver.cc:460] Webserver started at http://127.28.239.194:40013/ using document root <none> and password file <none>
I20260811 06:33:33.517952 21903 fs_manager.cc:359] Metadata directory not provided
I20260811 06:33:33.518177 21903 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:33:33.518620 21903 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:33:33.522840 21903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-1/data/instance:
uuid: "8c88b13171e84962a6de95aeeda55c2c"
format_stamp: "Formatted at 2026-08-11 06:33:33 on dist-test-slave-gkw7"
server_key: "49fbc2a75d3880e5bb58a29c326e0db9"
server_key_iv: "8e68b52c34c37c885ab9340e5c75e9cd"
server_key_version: "encryptionkey@0"
I20260811 06:33:33.523811 21903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-1/wal/instance:
uuid: "8c88b13171e84962a6de95aeeda55c2c"
format_stamp: "Formatted at 2026-08-11 06:33:33 on dist-test-slave-gkw7"
server_key: "49fbc2a75d3880e5bb58a29c326e0db9"
server_key_iv: "8e68b52c34c37c885ab9340e5c75e9cd"
server_key_version: "encryptionkey@0"
I20260811 06:33:33.532029 21903 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 06:33:33.535813 21924 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:33:33.537240 21903 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 06:33:33.537438 21903 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-1/wal
uuid: "8c88b13171e84962a6de95aeeda55c2c"
format_stamp: "Formatted at 2026-08-11 06:33:33 on dist-test-slave-gkw7"
server_key: "49fbc2a75d3880e5bb58a29c326e0db9"
server_key_iv: "8e68b52c34c37c885ab9340e5c75e9cd"
server_key_version: "encryptionkey@0"
I20260811 06:33:33.537722 21903 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:33:33.560452 21903 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:33:33.565442 21903 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:33:33.565812 21903 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:33:33.566969 21903 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:33:33.568475 21903 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:33:33.568585 21903 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:33:33.568747 21903 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:33:33.568826 21903 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:33:33.605732 21903 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.194:42877
I20260811 06:33:33.605855 22095 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.194:42877 every 8 connection(s)
I20260811 06:33:33.607373 21903 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
I20260811 06:33:33.617700 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 21903
I20260811 06:33:33.617924 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-1/wal/instance
I20260811 06:33:33.618453 29631 external_mini_cluster.cc:1428] Setting key 63d1e88d7712aacf917288b618442793
Aug 11 06:33:33 dist-test-slave-gkw7 krb5kdc[21544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430013, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.194@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:33:33.630118 22098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:33.610394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.194:37821 (local address 127.28.239.254:42189)
0811 06:33:33.610899 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0811 06:33:33.610904 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:33:33.612006 (+ 1102us) server_negotiation.cc:407] Connection header received
0811 06:33:33.614038 (+ 2032us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:33:33.614044 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:33:33.614142 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:33:33.614272 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:33:33.615877 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:33.616630 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:33.618485 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:33.618967 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:33.619109 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:33.623610 (+ 4501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:33:33.623615 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:33:33.623619 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:33:33.623652 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:33:33.626435 (+ 2783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:33:33.627565 (+ 1130us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:33:33.627568 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:33:33.627571 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:33:33.627703 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:33:33.628237 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:33:33.628240 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:33:33.628242 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:33:33.628513 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:33:33.628625 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 06:33:33.629674 (+ 1049us) server_negotiation.cc:299] Negotiation successful
0811 06:33:33.629852 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":255,"threads_started":1}
I20260811 06:33:33.631881 22099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:33.611074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:42189 (local address 127.28.239.194:37821)
0811 06:33:33.611827 (+ 753us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:33.611875 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 06:33:33.613733 (+ 1858us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:33:33.614539 (+ 806us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:33:33.614547 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:33:33.615096 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:33:33.615683 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:33:33.615701 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:33:33.616811 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:33:33.616815 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:33:33.618336 (+ 1521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:33:33.618350 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:33:33.619206 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:33:33.619213 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:33:33.619459 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:33.620273 (+ 814us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:33:33.620305 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:33:33.623354 (+ 3049us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:33:33.626678 (+ 3324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:33:33.626690 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:33:33.626709 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:33:33.627433 (+ 724us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:33:33.627815 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:33:33.627822 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:33:33.627829 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:33:33.628111 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:33:33.628647 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:33:33.628658 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:33:33.629211 (+ 553us) client_negotiation.cc:769] Sending connection context
0811 06:33:33.629559 (+ 348us) client_negotiation.cc:240] Negotiation successful
0811 06:33:33.630290 (+ 731us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":287,"threads_started":1}
I20260811 06:33:33.633513 22096 heartbeater.cc:347] Connected to a master server at 127.28.239.254:42189
I20260811 06:33:33.633895 22096 heartbeater.cc:464] Registering TS with master...
I20260811 06:33:33.634649 22096 heartbeater.cc:511] Master 127.28.239.254:42189 requested a full tablet report, sending...
I20260811 06:33:33.636236 21613 ts_manager.cc:194] Registered new tserver with Master: 8c88b13171e84962a6de95aeeda55c2c (127.28.239.194:42877)
I20260811 06:33:33.637132 21613 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.239.194@KRBTEST.COM'} at 127.28.239.194:37821
WARNING: no policy specified for kudu/127.28.239.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.239.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.239.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.195.keytab.
Entry for principal kudu/127.28.239.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.195.keytab.
I20260811 06:33:33.691273 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.195:0
--local_ip_for_outbound_sockets=127.28.239.195
--webserver_interface=127.28.239.195
--webserver_port=0
--tserver_master_addrs=127.28.239.254:42189
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.195.keytab
--principal=kudu/127.28.239.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:43401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 06:33:34.057660 21894 heartbeater.cc:503] Master 127.28.239.254:42189 was elected leader, sending a full tablet report...
W20260811 06:33:34.060459 22103 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:33:34.060770 22103 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:33:34.060876 22103 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:33:34.068782 22103 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:33:34.068928 22103 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:33:34.069063 22103 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.195
I20260811 06:33:34.080304 22103 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:43401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.195.keytab
--principal=kudu/127.28.239.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.239.195
--webserver_port=0
--tserver_master_addrs=127.28.239.254:42189
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.195
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:33:34.081723 22103 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:33:34.083376 22103 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:33:34.098218 22108 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:33:34.098259 22112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:33:34.098340 22109 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:33:34.099829 22103 server_base.cc:1034] running on GCE node
I20260811 06:33:34.100608 22103 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:33:34.101835 22103 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:33:34.103109 22103 hybrid_clock.cc:648] HybridClock initialized: now 1786430014103051 us; error 66 us; skew 500 ppm
I20260811 06:33:34.103490 22103 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:33:34 dist-test-slave-gkw7 krb5kdc[21544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430014, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:33:34.107381 22103 init.cc:377] Logged in from keytab as kudu/127.28.239.195@KRBTEST.COM (short username kudu)
I20260811 06:33:34.109020 22103 webserver.cc:460] Webserver started at http://127.28.239.195:41801/ using document root <none> and password file <none>
I20260811 06:33:34.109907 22103 fs_manager.cc:359] Metadata directory not provided
I20260811 06:33:34.110064 22103 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:33:34.110426 22103 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:33:34.113905 22103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-2/data/instance:
uuid: "1648c41809414c8f97307d271b2541d6"
format_stamp: "Formatted at 2026-08-11 06:33:34 on dist-test-slave-gkw7"
server_key: "9091360401284f9d006d7621f18df1ff"
server_key_iv: "0647b0b07517332b949870b3d5b6b892"
server_key_version: "encryptionkey@0"
I20260811 06:33:34.114840 22103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-2/wal/instance:
uuid: "1648c41809414c8f97307d271b2541d6"
format_stamp: "Formatted at 2026-08-11 06:33:34 on dist-test-slave-gkw7"
server_key: "9091360401284f9d006d7621f18df1ff"
server_key_iv: "0647b0b07517332b949870b3d5b6b892"
server_key_version: "encryptionkey@0"
I20260811 06:33:34.121381 22103 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260811 06:33:34.125393 22126 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:33:34.126930 22103 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:33:34.127139 22103 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-2/wal
uuid: "1648c41809414c8f97307d271b2541d6"
format_stamp: "Formatted at 2026-08-11 06:33:34 on dist-test-slave-gkw7"
server_key: "9091360401284f9d006d7621f18df1ff"
server_key_iv: "0647b0b07517332b949870b3d5b6b892"
server_key_version: "encryptionkey@0"
I20260811 06:33:34.127388 22103 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:33:34.146307 22103 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:33:34.151535 22103 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:33:34.151918 22103 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:33:34.153041 22103 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:33:34.154618 22103 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:33:34.154747 22103 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:33:34.154923 22103 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:33:34.155017 22103 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:33:34.192492 22103 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.195:45519
I20260811 06:33:34.192536 22284 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.195:45519 every 8 connection(s)
I20260811 06:33:34.194032 22103 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
I20260811 06:33:34.194619 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 22103
I20260811 06:33:34.194835 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-2/wal/instance
I20260811 06:33:34.195508 29631 external_mini_cluster.cc:1428] Setting key babb1c2e2b0265b72a475c0bdba7dbd5
Aug 11 06:33:34 dist-test-slave-gkw7 krb5kdc[21544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430014, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.195@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:33:34.217039 22287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:34.197178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.195:45477 (local address 127.28.239.254:42189)
0811 06:33:34.197758 (+ 580us) server_negotiation.cc:206] Beginning negotiation
0811 06:33:34.197942 (+ 184us) server_negotiation.cc:399] Waiting for connection header
0811 06:33:34.199069 (+ 1127us) server_negotiation.cc:407] Connection header received
0811 06:33:34.201253 (+ 2184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:33:34.201257 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:33:34.201357 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:33:34.201455 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:33:34.203141 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:34.203944 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:34.205754 (+ 1810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:34.206161 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:34.206263 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:34.210273 (+ 4010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:33:34.210281 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:33:34.210286 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:33:34.210335 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:33:34.213826 (+ 3491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:33:34.214564 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:33:34.214569 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:33:34.214571 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:33:34.214742 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:33:34.215243 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:33:34.215248 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:33:34.215251 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:33:34.215601 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:33:34.215752 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 06:33:34.216473 (+ 721us) server_negotiation.cc:299] Negotiation successful
0811 06:33:34.216727 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":436,"thread_start_us":239,"threads_started":1}
I20260811 06:33:34.217950 22288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:34.197829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:42189 (local address 127.28.239.195:45477)
0811 06:33:34.198848 (+ 1019us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:34.198910 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 06:33:34.200935 (+ 2025us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:33:34.201762 (+ 827us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:33:34.201771 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:33:34.202356 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:33:34.202986 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:33:34.203005 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:33:34.204099 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:33:34.204104 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:33:34.205523 (+ 1419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:33:34.205607 (+ 84us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:33:34.206373 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:33:34.206379 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:33:34.206596 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:34.207458 (+ 862us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:33:34.207483 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:33:34.210039 (+ 2556us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:33:34.213959 (+ 3920us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:33:34.213966 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:33:34.213979 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:33:34.214438 (+ 459us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:33:34.214840 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:33:34.214844 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:33:34.214846 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:33:34.215057 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:33:34.215733 (+ 676us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:33:34.215739 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:33:34.216073 (+ 334us) client_negotiation.cc:769] Sending connection context
0811 06:33:34.216315 (+ 242us) client_negotiation.cc:240] Negotiation successful
0811 06:33:34.216700 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":682,"thread_start_us":406,"threads_started":1}
I20260811 06:33:34.219403 22285 heartbeater.cc:347] Connected to a master server at 127.28.239.254:42189
I20260811 06:33:34.219749 22285 heartbeater.cc:464] Registering TS with master...
I20260811 06:33:34.220547 22285 heartbeater.cc:511] Master 127.28.239.254:42189 requested a full tablet report, sending...
I20260811 06:33:34.222122 21613 ts_manager.cc:194] Registered new tserver with Master: 1648c41809414c8f97307d271b2541d6 (127.28.239.195:45519)
I20260811 06:33:34.223030 21613 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.239.195@KRBTEST.COM'} at 127.28.239.195:45477
I20260811 06:33:34.224620 29631 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 06:33:34.243988 22287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:34.229182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41254 (local address 127.28.239.254:42189)
0811 06:33:34.229350 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 06:33:34.229354 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:33:34.229796 (+ 442us) server_negotiation.cc:407] Connection header received
0811 06:33:34.229940 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:33:34.229944 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:33:34.230003 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:33:34.230085 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:33:34.232062 (+ 1977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:34.233161 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:34.235039 (+ 1878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:34.235486 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:34.235588 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:34.238228 (+ 2640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:33:34.238233 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:33:34.238236 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:33:34.238268 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:33:34.240980 (+ 2712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:33:34.241878 (+ 898us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:33:34.241889 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:33:34.241891 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:33:34.242027 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:33:34.242446 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:33:34.242457 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:33:34.242460 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:33:34.242783 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:33:34.243072 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0811 06:33:34.243427 (+ 355us) server_negotiation.cc:299] Negotiation successful
0811 06:33:34.243661 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 06:33:34.248533 21613 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41254:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:33:34.256191 21613 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:33:34.284160 22310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:34.270268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.195:45519 (local address 127.0.0.1:36720)
0811 06:33:34.271993 (+ 1725us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:34.272042 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 06:33:34.272249 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:33:34.273494 (+ 1245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:33:34.273506 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:33:34.273577 (+ 71us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:33:34.274164 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:33:34.274183 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:33:34.276710 (+ 2527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:33:34.276719 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:33:34.279788 (+ 3069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:33:34.279812 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:33:34.282739 (+ 2927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:33:34.282746 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:33:34.283050 (+ 304us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:34.283131 (+ 81us) client_negotiation.cc:769] Sending connection context
0811 06:33:34.283307 (+ 176us) client_negotiation.cc:240] Negotiation successful
0811 06:33:34.283583 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1468,"threads_started":1}
I20260811 06:33:34.285398 22307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:34.270418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36720 (local address 127.28.239.195:45519)
0811 06:33:34.271354 (+ 936us) server_negotiation.cc:206] Beginning negotiation
0811 06:33:34.271369 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 06:33:34.272468 (+ 1099us) server_negotiation.cc:407] Connection header received
0811 06:33:34.272599 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:33:34.272610 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:33:34.272884 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:33:34.273110 (+ 226us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:33:34.274433 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:34.276482 (+ 2049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:34.280065 (+ 3583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:34.282101 (+ 2036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:34.282362 (+ 261us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:34.282586 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0811 06:33:34.284649 (+ 2063us) server_negotiation.cc:299] Negotiation successful
0811 06:33:34.284895 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":666,"thread_start_us":1043,"threads_started":1}
I20260811 06:33:34.292409 22197 tablet_service.cc:1467] Processing CreateTablet for tablet 34fd9c6c04394c48ba3bd6b8e937184f (DEFAULT_TABLE table=test-table [id=70ddf8e549dd420e87d312014b328e85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:33:34.294275 22197 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34fd9c6c04394c48ba3bd6b8e937184f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:33:34.294992 22310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:34.270463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.193:42485 (local address 127.0.0.1:49820)
0811 06:33:34.285949 (+ 15486us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:34.285978 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0811 06:33:34.286129 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:33:34.288625 (+ 2496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:33:34.288630 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:33:34.288660 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:33:34.288886 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:33:34.288904 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:33:34.290633 (+ 1729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:33:34.290638 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:33:34.292559 (+ 1921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:33:34.292575 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:33:34.294091 (+ 1516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:33:34.294098 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:33:34.294312 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:34.294356 (+ 44us) client_negotiation.cc:769] Sending connection context
0811 06:33:34.294469 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 06:33:34.294635 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15300,"thread_start_us":1124,"threads_started":1}
I20260811 06:33:34.295083 22312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:34.271214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49820 (local address 127.28.239.193:42485)
0811 06:33:34.288028 (+ 16814us) server_negotiation.cc:206] Beginning negotiation
0811 06:33:34.288048 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 06:33:34.288081 (+ 33us) server_negotiation.cc:407] Connection header received
0811 06:33:34.288210 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:33:34.288217 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:33:34.288401 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:33:34.288644 (+ 243us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:33:34.289115 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:34.290447 (+ 1332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:34.292744 (+ 2297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:34.293889 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:34.294095 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:34.294289 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0811 06:33:34.294542 (+ 253us) server_negotiation.cc:299] Negotiation successful
0811 06:33:34.294716 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16615,"thread_start_us":16412,"threads_started":1}
I20260811 06:33:34.297194 22304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:34.272445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.194:42877 (local address 127.0.0.1:45936)
0811 06:33:34.286923 (+ 14478us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:34.286941 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 06:33:34.287061 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:33:34.288884 (+ 1823us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:33:34.288891 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:33:34.288926 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:33:34.289332 (+ 406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:33:34.289344 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:33:34.291479 (+ 2135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:33:34.291486 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:33:34.293744 (+ 2258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:33:34.293763 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:33:34.296417 (+ 2654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:33:34.296421 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:33:34.296542 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:34.296566 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 06:33:34.296688 (+ 122us) client_negotiation.cc:240] Negotiation successful
0811 06:33:34.296851 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14349,"thread_start_us":14844,"threads_started":1}
I20260811 06:33:34.298102 22316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:34.272985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45936 (local address 127.28.239.194:42877)
0811 06:33:34.286972 (+ 13987us) server_negotiation.cc:206] Beginning negotiation
0811 06:33:34.286987 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 06:33:34.287050 (+ 63us) server_negotiation.cc:407] Connection header received
0811 06:33:34.288050 (+ 1000us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:33:34.288060 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:33:34.288358 (+ 298us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:33:34.288542 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:33:34.289585 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:34.291280 (+ 1695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:34.293952 (+ 2672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:34.295852 (+ 1900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:34.296016 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:34.296197 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0811 06:33:34.297578 (+ 1381us) server_negotiation.cc:299] Negotiation successful
0811 06:33:34.297781 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13778,"thread_start_us":13613,"threads_started":1}
I20260811 06:33:34.299717 21796 tablet_service.cc:1467] Processing CreateTablet for tablet 34fd9c6c04394c48ba3bd6b8e937184f (DEFAULT_TABLE table=test-table [id=70ddf8e549dd420e87d312014b328e85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:33:34.301898 21796 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34fd9c6c04394c48ba3bd6b8e937184f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:33:34.302188 22000 tablet_service.cc:1467] Processing CreateTablet for tablet 34fd9c6c04394c48ba3bd6b8e937184f (DEFAULT_TABLE table=test-table [id=70ddf8e549dd420e87d312014b328e85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:33:34.303668 22000 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34fd9c6c04394c48ba3bd6b8e937184f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:33:34.310393 22317 tablet_bootstrap.cc:492] T 34fd9c6c04394c48ba3bd6b8e937184f P 1648c41809414c8f97307d271b2541d6: Bootstrap starting.
I20260811 06:33:34.314637 22317 tablet_bootstrap.cc:654] T 34fd9c6c04394c48ba3bd6b8e937184f P 1648c41809414c8f97307d271b2541d6: Neither blocks nor log segments found. Creating new log.
I20260811 06:33:34.316289 22317 log.cc:824] T 34fd9c6c04394c48ba3bd6b8e937184f P 1648c41809414c8f97307d271b2541d6: Log is configured to *not* fsync() on all Append() calls
I20260811 06:33:34.318018 22319 tablet_bootstrap.cc:492] T 34fd9c6c04394c48ba3bd6b8e937184f P 5e51526ceea347ba88b10bc1a50d84ca: Bootstrap starting.
I20260811 06:33:34.319933 22322 tablet_bootstrap.cc:492] T 34fd9c6c04394c48ba3bd6b8e937184f P 8c88b13171e84962a6de95aeeda55c2c: Bootstrap starting.
I20260811 06:33:34.320415 22317 tablet_bootstrap.cc:492] T 34fd9c6c04394c48ba3bd6b8e937184f P 1648c41809414c8f97307d271b2541d6: No bootstrap required, opened a new log
I20260811 06:33:34.320724 22317 ts_tablet_manager.cc:1397] T 34fd9c6c04394c48ba3bd6b8e937184f P 1648c41809414c8f97307d271b2541d6: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260811 06:33:34.322508 22319 tablet_bootstrap.cc:654] T 34fd9c6c04394c48ba3bd6b8e937184f P 5e51526ceea347ba88b10bc1a50d84ca: Neither blocks nor log segments found. Creating new log.
I20260811 06:33:34.323398 22322 tablet_bootstrap.cc:654] T 34fd9c6c04394c48ba3bd6b8e937184f P 8c88b13171e84962a6de95aeeda55c2c: Neither blocks nor log segments found. Creating new log.
I20260811 06:33:34.323864 22319 log.cc:824] T 34fd9c6c04394c48ba3bd6b8e937184f P 5e51526ceea347ba88b10bc1a50d84ca: Log is configured to *not* fsync() on all Append() calls
I20260811 06:33:34.324577 22322 log.cc:824] T 34fd9c6c04394c48ba3bd6b8e937184f P 8c88b13171e84962a6de95aeeda55c2c: Log is configured to *not* fsync() on all Append() calls
I20260811 06:33:34.327405 22319 tablet_bootstrap.cc:492] T 34fd9c6c04394c48ba3bd6b8e937184f P 5e51526ceea347ba88b10bc1a50d84ca: No bootstrap required, opened a new log
I20260811 06:33:34.327706 22319 ts_tablet_manager.cc:1397] T 34fd9c6c04394c48ba3bd6b8e937184f P 5e51526ceea347ba88b10bc1a50d84ca: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.003s
I20260811 06:33:34.330381 22322 tablet_bootstrap.cc:492] T 34fd9c6c04394c48ba3bd6b8e937184f P 8c88b13171e84962a6de95aeeda55c2c: No bootstrap required, opened a new log
I20260811 06:33:34.330744 22322 ts_tablet_manager.cc:1397] T 34fd9c6c04394c48ba3bd6b8e937184f P 8c88b13171e84962a6de95aeeda55c2c: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260811 06:33:34.331089 22317 raft_consensus.cc:348] T 34fd9c6c04394c48ba3bd6b8e937184f P 1648c41809414c8f97307d271b2541d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1648c41809414c8f97307d271b2541d6" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 45519 } } peers { permanent_uuid: "5e51526ceea347ba88b10bc1a50d84ca" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 42485 } } peers { permanent_uuid: "8c88b13171e84962a6de95aeeda55c2c" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 42877 } }
I20260811 06:33:34.331627 22317 raft_consensus.cc:374] T 34fd9c6c04394c48ba3bd6b8e937184f P 1648c41809414c8f97307d271b2541d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:33:34.331748 22317 raft_consensus.cc:729] T 34fd9c6c04394c48ba3bd6b8e937184f P 1648c41809414c8f97307d271b2541d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1648c41809414c8f97307d271b2541d6, State: Initialized, Role: FOLLOWER
I20260811 06:33:34.332501 22317 consensus_queue.cc:260] T 34fd9c6c04394c48ba3bd6b8e937184f P 1648c41809414c8f97307d271b2541d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1648c41809414c8f97307d271b2541d6" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 45519 } } peers { permanent_uuid: "5e51526ceea347ba88b10bc1a50d84ca" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 42485 } } peers { permanent_uuid: "8c88b13171e84962a6de95aeeda55c2c" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 42877 } }
I20260811 06:33:34.333792 22285 heartbeater.cc:503] Master 127.28.239.254:42189 was elected leader, sending a full tablet report...
I20260811 06:33:34.334566 22317 ts_tablet_manager.cc:1428] T 34fd9c6c04394c48ba3bd6b8e937184f P 1648c41809414c8f97307d271b2541d6: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260811 06:33:34.337495 22319 raft_consensus.cc:348] T 34fd9c6c04394c48ba3bd6b8e937184f P 5e51526ceea347ba88b10bc1a50d84ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1648c41809414c8f97307d271b2541d6" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 45519 } } peers { permanent_uuid: "5e51526ceea347ba88b10bc1a50d84ca" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 42485 } } peers { permanent_uuid: "8c88b13171e84962a6de95aeeda55c2c" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 42877 } }
I20260811 06:33:34.337942 22319 raft_consensus.cc:374] T 34fd9c6c04394c48ba3bd6b8e937184f P 5e51526ceea347ba88b10bc1a50d84ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:33:34.338106 22319 raft_consensus.cc:729] T 34fd9c6c04394c48ba3bd6b8e937184f P 5e51526ceea347ba88b10bc1a50d84ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e51526ceea347ba88b10bc1a50d84ca, State: Initialized, Role: FOLLOWER
I20260811 06:33:34.338757 22319 consensus_queue.cc:260] T 34fd9c6c04394c48ba3bd6b8e937184f P 5e51526ceea347ba88b10bc1a50d84ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1648c41809414c8f97307d271b2541d6" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 45519 } } peers { permanent_uuid: "5e51526ceea347ba88b10bc1a50d84ca" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 42485 } } peers { permanent_uuid: "8c88b13171e84962a6de95aeeda55c2c" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 42877 } }
I20260811 06:33:34.342051 22319 ts_tablet_manager.cc:1428] T 34fd9c6c04394c48ba3bd6b8e937184f P 5e51526ceea347ba88b10bc1a50d84ca: Time spent starting tablet: real 0.014s user 0.014s sys 0.001s
I20260811 06:33:34.343029 22322 raft_consensus.cc:348] T 34fd9c6c04394c48ba3bd6b8e937184f P 8c88b13171e84962a6de95aeeda55c2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1648c41809414c8f97307d271b2541d6" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 45519 } } peers { permanent_uuid: "5e51526ceea347ba88b10bc1a50d84ca" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 42485 } } peers { permanent_uuid: "8c88b13171e84962a6de95aeeda55c2c" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 42877 } }
I20260811 06:33:34.343449 22322 raft_consensus.cc:374] T 34fd9c6c04394c48ba3bd6b8e937184f P 8c88b13171e84962a6de95aeeda55c2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:33:34.343566 22322 raft_consensus.cc:729] T 34fd9c6c04394c48ba3bd6b8e937184f P 8c88b13171e84962a6de95aeeda55c2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c88b13171e84962a6de95aeeda55c2c, State: Initialized, Role: FOLLOWER
I20260811 06:33:34.344280 22322 consensus_queue.cc:260] T 34fd9c6c04394c48ba3bd6b8e937184f P 8c88b13171e84962a6de95aeeda55c2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1648c41809414c8f97307d271b2541d6" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 45519 } } peers { permanent_uuid: "5e51526ceea347ba88b10bc1a50d84ca" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 42485 } } peers { permanent_uuid: "8c88b13171e84962a6de95aeeda55c2c" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 42877 } }
I20260811 06:33:34.345504 22096 heartbeater.cc:503] Master 127.28.239.254:42189 was elected leader, sending a full tablet report...
I20260811 06:33:34.346294 22322 ts_tablet_manager.cc:1428] T 34fd9c6c04394c48ba3bd6b8e937184f P 8c88b13171e84962a6de95aeeda55c2c: Time spent starting tablet: real 0.015s user 0.016s sys 0.000s
W20260811 06:33:34.359517 22097 tablet.cc:2369] T 34fd9c6c04394c48ba3bd6b8e937184f P 8c88b13171e84962a6de95aeeda55c2c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 06:33:34.446007 22286 tablet.cc:2369] T 34fd9c6c04394c48ba3bd6b8e937184f P 1648c41809414c8f97307d271b2541d6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 06:33:34.522778 21895 tablet.cc:2369] T 34fd9c6c04394c48ba3bd6b8e937184f P 5e51526ceea347ba88b10bc1a50d84ca: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:33:34.603873 22325 raft_consensus.cc:482] T 34fd9c6c04394c48ba3bd6b8e937184f P 1648c41809414c8f97307d271b2541d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:33:34.604288 22325 raft_consensus.cc:504] T 34fd9c6c04394c48ba3bd6b8e937184f P 1648c41809414c8f97307d271b2541d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1648c41809414c8f97307d271b2541d6" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 45519 } } peers { permanent_uuid: "5e51526ceea347ba88b10bc1a50d84ca" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 42485 } } peers { permanent_uuid: "8c88b13171e84962a6de95aeeda55c2c" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 42877 } }
I20260811 06:33:34.606336 22325 leader_election.cc:288] T 34fd9c6c04394c48ba3bd6b8e937184f P 1648c41809414c8f97307d271b2541d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e51526ceea347ba88b10bc1a50d84ca (127.28.239.193:42485), 8c88b13171e84962a6de95aeeda55c2c (127.28.239.194:42877)
I20260811 06:33:34.612802 22288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:34.606652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.193:42485 (local address 127.28.239.195:49355)
0811 06:33:34.606884 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:34.606909 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 06:33:34.607074 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:33:34.607688 (+ 614us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:33:34.607693 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:33:34.607722 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:33:34.607980 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:33:34.607993 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:33:34.609471 (+ 1478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:33:34.609480 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:33:34.610908 (+ 1428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:33:34.610924 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:33:34.612031 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:33:34.612037 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:33:34.612238 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:34.612264 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 06:33:34.612370 (+ 106us) client_negotiation.cc:240] Negotiation successful
0811 06:33:34.612509 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71}
I20260811 06:33:34.612782 22312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:34.606810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.195:49355 (local address 127.28.239.193:42485)
0811 06:33:34.607026 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 06:33:34.607033 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:33:34.607056 (+ 23us) server_negotiation.cc:407] Connection header received
0811 06:33:34.607300 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:33:34.607306 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:33:34.607401 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:33:34.607496 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:33:34.608206 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:34.609320 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:34.611072 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:34.611867 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:34.612036 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:34.612114 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 06:33:34.612400 (+ 286us) server_negotiation.cc:299] Negotiation successful
0811 06:33:34.612531 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 06:33:34.614006 21825 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34fd9c6c04394c48ba3bd6b8e937184f" candidate_uuid: "1648c41809414c8f97307d271b2541d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e51526ceea347ba88b10bc1a50d84ca" is_pre_election: true
I20260811 06:33:34.614578 21825 raft_consensus.cc:2454] T 34fd9c6c04394c48ba3bd6b8e937184f P 5e51526ceea347ba88b10bc1a50d84ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1648c41809414c8f97307d271b2541d6 in term 0.
I20260811 06:33:34.615403 22316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:34.607501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.195:35831 (local address 127.28.239.194:42877)
0811 06:33:34.607724 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0811 06:33:34.607729 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:33:34.608381 (+ 652us) server_negotiation.cc:407] Connection header received
0811 06:33:34.608532 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:33:34.608538 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:33:34.608611 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:33:34.608703 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:33:34.609330 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:34.610476 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:34.612964 (+ 2488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:34.614321 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:34.614461 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:34.614527 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 06:33:34.614954 (+ 427us) server_negotiation.cc:299] Negotiation successful
0811 06:33:34.615146 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 06:33:34.615427 22328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:34.607035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.194:42877 (local address 127.28.239.195:35831)
0811 06:33:34.608256 (+ 1221us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:34.608276 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 06:33:34.608401 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:33:34.608752 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:33:34.608756 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:33:34.608782 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:33:34.609150 (+ 368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:33:34.609162 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:33:34.610678 (+ 1516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:33:34.610683 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:33:34.612776 (+ 2093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:33:34.612794 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:33:34.614500 (+ 1706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:33:34.614506 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:33:34.614731 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:34.614767 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 06:33:34.614916 (+ 149us) client_negotiation.cc:240] Negotiation successful
0811 06:33:34.615052 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1107,"thread_start_us":915,"threads_started":1}
I20260811 06:33:34.615482 22132 leader_election.cc:302] T 34fd9c6c04394c48ba3bd6b8e937184f P 1648c41809414c8f97307d271b2541d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1648c41809414c8f97307d271b2541d6, 5e51526ceea347ba88b10bc1a50d84ca; no voters:
I20260811 06:33:34.616077 22325 raft_consensus.cc:2781] T 34fd9c6c04394c48ba3bd6b8e937184f P 1648c41809414c8f97307d271b2541d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:33:34.616242 22325 raft_consensus.cc:482] T 34fd9c6c04394c48ba3bd6b8e937184f P 1648c41809414c8f97307d271b2541d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:33:34.616353 22325 raft_consensus.cc:3037] T 34fd9c6c04394c48ba3bd6b8e937184f P 1648c41809414c8f97307d271b2541d6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:33:34.616755 22032 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34fd9c6c04394c48ba3bd6b8e937184f" candidate_uuid: "1648c41809414c8f97307d271b2541d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c88b13171e84962a6de95aeeda55c2c" is_pre_election: true
I20260811 06:33:34.617267 22032 raft_consensus.cc:2454] T 34fd9c6c04394c48ba3bd6b8e937184f P 8c88b13171e84962a6de95aeeda55c2c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1648c41809414c8f97307d271b2541d6 in term 0.
I20260811 06:33:34.619138 22325 raft_consensus.cc:504] T 34fd9c6c04394c48ba3bd6b8e937184f P 1648c41809414c8f97307d271b2541d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1648c41809414c8f97307d271b2541d6" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 45519 } } peers { permanent_uuid: "5e51526ceea347ba88b10bc1a50d84ca" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 42485 } } peers { permanent_uuid: "8c88b13171e84962a6de95aeeda55c2c" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 42877 } }
I20260811 06:33:34.619797 22325 leader_election.cc:288] T 34fd9c6c04394c48ba3bd6b8e937184f P 1648c41809414c8f97307d271b2541d6 [CANDIDATE]: Term 1 election: Requested vote from peers 5e51526ceea347ba88b10bc1a50d84ca (127.28.239.193:42485), 8c88b13171e84962a6de95aeeda55c2c (127.28.239.194:42877)
I20260811 06:33:34.620283 21825 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34fd9c6c04394c48ba3bd6b8e937184f" candidate_uuid: "1648c41809414c8f97307d271b2541d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e51526ceea347ba88b10bc1a50d84ca"
I20260811 06:33:34.620353 22032 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34fd9c6c04394c48ba3bd6b8e937184f" candidate_uuid: "1648c41809414c8f97307d271b2541d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c88b13171e84962a6de95aeeda55c2c"
I20260811 06:33:34.620527 21825 raft_consensus.cc:3037] T 34fd9c6c04394c48ba3bd6b8e937184f P 5e51526ceea347ba88b10bc1a50d84ca [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:33:34.620568 22032 raft_consensus.cc:3037] T 34fd9c6c04394c48ba3bd6b8e937184f P 8c88b13171e84962a6de95aeeda55c2c [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:33:34.622946 21825 raft_consensus.cc:2454] T 34fd9c6c04394c48ba3bd6b8e937184f P 5e51526ceea347ba88b10bc1a50d84ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1648c41809414c8f97307d271b2541d6 in term 1.
I20260811 06:33:34.623034 22032 raft_consensus.cc:2454] T 34fd9c6c04394c48ba3bd6b8e937184f P 8c88b13171e84962a6de95aeeda55c2c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1648c41809414c8f97307d271b2541d6 in term 1.
I20260811 06:33:34.623440 22132 leader_election.cc:302] T 34fd9c6c04394c48ba3bd6b8e937184f P 1648c41809414c8f97307d271b2541d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1648c41809414c8f97307d271b2541d6, 5e51526ceea347ba88b10bc1a50d84ca; no voters:
I20260811 06:33:34.623768 22325 raft_consensus.cc:2781] T 34fd9c6c04394c48ba3bd6b8e937184f P 1648c41809414c8f97307d271b2541d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:33:34.624240 22325 raft_consensus.cc:686] T 34fd9c6c04394c48ba3bd6b8e937184f P 1648c41809414c8f97307d271b2541d6 [term 1 LEADER]: Becoming Leader. State: Replica: 1648c41809414c8f97307d271b2541d6, State: Running, Role: LEADER
I20260811 06:33:34.624840 22325 consensus_queue.cc:237] T 34fd9c6c04394c48ba3bd6b8e937184f P 1648c41809414c8f97307d271b2541d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1648c41809414c8f97307d271b2541d6" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 45519 } } peers { permanent_uuid: "5e51526ceea347ba88b10bc1a50d84ca" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 42485 } } peers { permanent_uuid: "8c88b13171e84962a6de95aeeda55c2c" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 42877 } }
I20260811 06:33:34.630007 21612 catalog_manager.cc:5495] T 34fd9c6c04394c48ba3bd6b8e937184f P 1648c41809414c8f97307d271b2541d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1648c41809414c8f97307d271b2541d6 (127.28.239.195). New cstate: current_term: 1 leader_uuid: "1648c41809414c8f97307d271b2541d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1648c41809414c8f97307d271b2541d6" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 45519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e51526ceea347ba88b10bc1a50d84ca" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 42485 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c88b13171e84962a6de95aeeda55c2c" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 42877 } health_report { overall_health: UNKNOWN } } }
I20260811 06:33:34.669633 22307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:34.663701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36736 (local address 127.28.239.195:45519)
0811 06:33:34.663934 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0811 06:33:34.663939 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:33:34.663997 (+ 58us) server_negotiation.cc:407] Connection header received
0811 06:33:34.664222 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:33:34.664226 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:33:34.664306 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:33:34.664433 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:33:34.664829 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:34.665688 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:34.667377 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:34.667784 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:34.667881 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:34.668389 (+ 508us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:33:34.668944 (+ 555us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:33:34.669052 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 06:33:34.669202 (+ 150us) server_negotiation.cc:299] Negotiation successful
0811 06:33:34.669269 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
I20260811 06:33:34.678308 21825 raft_consensus.cc:1261] T 34fd9c6c04394c48ba3bd6b8e937184f P 5e51526ceea347ba88b10bc1a50d84ca [term 1 FOLLOWER]: Refusing update from remote peer 1648c41809414c8f97307d271b2541d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:33:34.678352 22032 raft_consensus.cc:1261] T 34fd9c6c04394c48ba3bd6b8e937184f P 8c88b13171e84962a6de95aeeda55c2c [term 1 FOLLOWER]: Refusing update from remote peer 1648c41809414c8f97307d271b2541d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:33:34.679314 22329 consensus_queue.cc:1035] T 34fd9c6c04394c48ba3bd6b8e937184f P 1648c41809414c8f97307d271b2541d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e51526ceea347ba88b10bc1a50d84ca" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 42485 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:33:34.679741 22325 consensus_queue.cc:1035] T 34fd9c6c04394c48ba3bd6b8e937184f P 1648c41809414c8f97307d271b2541d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c88b13171e84962a6de95aeeda55c2c" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 42877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:33:34.696105 22335 mvcc.cc:205] Tried to move back new op lower bound from 7317217340112158720 to 7317217339907641344. Current Snapshot: MvccSnapshot[applied={T|T < 7317217340112158720}]
I20260811 06:33:34.714262 21612 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41254:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:33:34.714565 21612 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41254:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:33:34.718381 21612 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f9315cc436a5414ea115e916586d7497: Sending DeleteTablet for 3 replicas of tablet 34fd9c6c04394c48ba3bd6b8e937184f
I20260811 06:33:34.719832 21796 tablet_service.cc:1514] Processing DeleteTablet for tablet 34fd9c6c04394c48ba3bd6b8e937184f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:33:34 UTC) from {username='kudu', principal='kudu/127.28.239.254@KRBTEST.COM'} at 127.0.0.1:49820
I20260811 06:33:34.720103 22197 tablet_service.cc:1514] Processing DeleteTablet for tablet 34fd9c6c04394c48ba3bd6b8e937184f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:33:34 UTC) from {username='kudu', principal='kudu/127.28.239.254@KRBTEST.COM'} at 127.0.0.1:36720
I20260811 06:33:34.720093 22000 tablet_service.cc:1514] Processing DeleteTablet for tablet 34fd9c6c04394c48ba3bd6b8e937184f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:33:34 UTC) from {username='kudu', principal='kudu/127.28.239.254@KRBTEST.COM'} at 127.0.0.1:45936
I20260811 06:33:34.721576 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 21711
I20260811 06:33:34.726276 22347 tablet_replica.cc:330] T 34fd9c6c04394c48ba3bd6b8e937184f P 1648c41809414c8f97307d271b2541d6: stopping tablet replica
I20260811 06:33:34.726920 22347 raft_consensus.cc:2229] T 34fd9c6c04394c48ba3bd6b8e937184f P 1648c41809414c8f97307d271b2541d6 [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:33:34.734414 22347 raft_consensus.cc:2258] T 34fd9c6c04394c48ba3bd6b8e937184f P 1648c41809414c8f97307d271b2541d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:33:34.735180 22345 tablet_replica.cc:330] T 34fd9c6c04394c48ba3bd6b8e937184f P 5e51526ceea347ba88b10bc1a50d84ca: stopping tablet replica
I20260811 06:33:34.735374 22346 tablet_replica.cc:330] T 34fd9c6c04394c48ba3bd6b8e937184f P 8c88b13171e84962a6de95aeeda55c2c: stopping tablet replica
I20260811 06:33:34.735733 22345 raft_consensus.cc:2229] T 34fd9c6c04394c48ba3bd6b8e937184f P 5e51526ceea347ba88b10bc1a50d84ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:33:34.735780 22346 raft_consensus.cc:2229] T 34fd9c6c04394c48ba3bd6b8e937184f P 8c88b13171e84962a6de95aeeda55c2c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:33:34.736349 22346 raft_consensus.cc:2258] T 34fd9c6c04394c48ba3bd6b8e937184f P 8c88b13171e84962a6de95aeeda55c2c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:33:34.736349 22345 raft_consensus.cc:2258] T 34fd9c6c04394c48ba3bd6b8e937184f P 5e51526ceea347ba88b10bc1a50d84ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:33:34.738566 22347 ts_tablet_manager.cc:1905] T 34fd9c6c04394c48ba3bd6b8e937184f P 1648c41809414c8f97307d271b2541d6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:33:34.739025 22345 ts_tablet_manager.cc:1905] T 34fd9c6c04394c48ba3bd6b8e937184f P 5e51526ceea347ba88b10bc1a50d84ca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:33:34.741016 22346 ts_tablet_manager.cc:1905] T 34fd9c6c04394c48ba3bd6b8e937184f P 8c88b13171e84962a6de95aeeda55c2c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:33:34.746872 22345 ts_tablet_manager.cc:1918] T 34fd9c6c04394c48ba3bd6b8e937184f P 5e51526ceea347ba88b10bc1a50d84ca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:33:34.747067 22345 log.cc:1196] T 34fd9c6c04394c48ba3bd6b8e937184f P 5e51526ceea347ba88b10bc1a50d84ca: Deleting WAL directory at /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-0/wal/wals/34fd9c6c04394c48ba3bd6b8e937184f
I20260811 06:33:34.747503 22347 ts_tablet_manager.cc:1918] T 34fd9c6c04394c48ba3bd6b8e937184f P 1648c41809414c8f97307d271b2541d6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:33:34.747699 22345 ts_tablet_manager.cc:1939] T 34fd9c6c04394c48ba3bd6b8e937184f P 5e51526ceea347ba88b10bc1a50d84ca: Deleting consensus metadata
I20260811 06:33:34.747813 22347 log.cc:1196] T 34fd9c6c04394c48ba3bd6b8e937184f P 1648c41809414c8f97307d271b2541d6: Deleting WAL directory at /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-2/wal/wals/34fd9c6c04394c48ba3bd6b8e937184f
I20260811 06:33:34.748502 22347 ts_tablet_manager.cc:1939] T 34fd9c6c04394c48ba3bd6b8e937184f P 1648c41809414c8f97307d271b2541d6: Deleting consensus metadata
I20260811 06:33:34.748843 22346 ts_tablet_manager.cc:1918] T 34fd9c6c04394c48ba3bd6b8e937184f P 8c88b13171e84962a6de95aeeda55c2c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:33:34.749044 22346 log.cc:1196] T 34fd9c6c04394c48ba3bd6b8e937184f P 8c88b13171e84962a6de95aeeda55c2c: Deleting WAL directory at /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-1/wal/wals/34fd9c6c04394c48ba3bd6b8e937184f
I20260811 06:33:34.749935 21593 catalog_manager.cc:4841] TS 5e51526ceea347ba88b10bc1a50d84ca (127.28.239.193:42485): tablet 34fd9c6c04394c48ba3bd6b8e937184f (table test-table [id=70ddf8e549dd420e87d312014b328e85]) successfully deleted
I20260811 06:33:34.750469 21587 catalog_manager.cc:4841] TS 1648c41809414c8f97307d271b2541d6 (127.28.239.195:45519): tablet 34fd9c6c04394c48ba3bd6b8e937184f (table test-table [id=70ddf8e549dd420e87d312014b328e85]) successfully deleted
I20260811 06:33:34.752565 22346 ts_tablet_manager.cc:1939] T 34fd9c6c04394c48ba3bd6b8e937184f P 8c88b13171e84962a6de95aeeda55c2c: Deleting consensus metadata
I20260811 06:33:34.754379 21593 catalog_manager.cc:4841] TS 8c88b13171e84962a6de95aeeda55c2c (127.28.239.194:42877): tablet 34fd9c6c04394c48ba3bd6b8e937184f (table test-table [id=70ddf8e549dd420e87d312014b328e85]) successfully deleted
Aug 11 06:33:34 dist-test-slave-gkw7 krb5kdc[21544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430011, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.239.193@KRBTEST.COM
I20260811 06:33:34.769956 22312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:34.723192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49824 (local address 127.28.239.193:42485)
0811 06:33:34.723431 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0811 06:33:34.723440 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:33:34.735423 (+ 11983us) server_negotiation.cc:407] Connection header received
0811 06:33:34.735675 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:33:34.735681 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:33:34.735775 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:33:34.735889 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:33:34.749703 (+ 13814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:34.750845 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:34.752500 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:34.753125 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:34.753290 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:34.762600 (+ 9310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:33:34.762619 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:33:34.762641 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:33:34.762695 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:33:34.766780 (+ 4085us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:33:34.767531 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:33:34.767543 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:33:34.767549 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:33:34.767723 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:33:34.768125 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:33:34.768140 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:33:34.768143 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:33:34.768626 (+ 483us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:33:34.768935 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0811 06:33:34.769229 (+ 294us) server_negotiation.cc:299] Negotiation successful
0811 06:33:34.769465 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 06:33:34.770752 21886 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:33:34.902896 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 21711
I20260811 06:33:34.928606 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 21903
Aug 11 06:33:34 dist-test-slave-gkw7 krb5kdc[21544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430011, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.239.194@KRBTEST.COM
I20260811 06:33:34.946218 22316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:34.929510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45948 (local address 127.28.239.194:42877)
0811 06:33:34.929827 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0811 06:33:34.929834 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:33:34.929858 (+ 24us) server_negotiation.cc:407] Connection header received
0811 06:33:34.929930 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:33:34.929936 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:33:34.930028 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:33:34.930143 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:33:34.931690 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:34.932456 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:34.934015 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:34.934423 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:34.934561 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:34.939295 (+ 4734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:33:34.939318 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:33:34.939340 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:33:34.939384 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:33:34.943277 (+ 3893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:33:34.943994 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:33:34.944003 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:33:34.944009 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:33:34.944163 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:33:34.944538 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:33:34.944543 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:33:34.944547 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:33:34.945042 (+ 495us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:33:34.945353 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0811 06:33:34.945577 (+ 224us) server_negotiation.cc:299] Negotiation successful
0811 06:33:34.945846 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165}
I20260811 06:33:34.946990 22090 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:33:35.039049 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 21903
I20260811 06:33:35.072450 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 22103
Aug 11 06:33:35 dist-test-slave-gkw7 krb5kdc[21544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430011, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.239.195@KRBTEST.COM
I20260811 06:33:35.090065 22307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:35.073489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36750 (local address 127.28.239.195:45519)
0811 06:33:35.073770 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0811 06:33:35.073779 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:33:35.073802 (+ 23us) server_negotiation.cc:407] Connection header received
0811 06:33:35.073887 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:33:35.073891 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:33:35.073954 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:33:35.074084 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:33:35.075576 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:35.076445 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:35.078145 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:35.078623 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:35.078773 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:35.083380 (+ 4607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:33:35.083394 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:33:35.083411 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:33:35.083459 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:33:35.086900 (+ 3441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:33:35.087702 (+ 802us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:33:35.087716 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:33:35.087719 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:33:35.087897 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:33:35.088321 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:33:35.088325 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:33:35.088328 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:33:35.088730 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:33:35.089097 (+ 367us) server_negotiation.cc:1035] Waiting for connection context
0811 06:33:35.089411 (+ 314us) server_negotiation.cc:299] Negotiation successful
0811 06:33:35.089680 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143}
I20260811 06:33:35.090950 22279 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:33:35.216581 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 22103
I20260811 06:33:35.249411 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 21565
I20260811 06:33:35.250615 21662 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:33:35.366560 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 21565
I20260811 06:33:55.400195 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:42189
--webserver_interface=127.28.239.254
--webserver_port=37769
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab
--principal=kudu/127.28.239.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:43401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.239.254:42189
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:33:55.759907 22354 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:33:55.760195 22354 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:33:55.760352 22354 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:33:55.767853 22354 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 06:33:55.767956 22354 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 06:33:55.768031 22354 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 06:33:55.768122 22354 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:33:55.768219 22354 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:33:55.768335 22354 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:33:55.768407 22354 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:33:55.768488 22354 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:33:55.778743 22354 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:43401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.28.239.254:42189
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab
--principal=kudu/127.28.239.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:42189
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.239.254
--webserver_port=37769
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:33:55.780187 22354 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:33:55.781713 22354 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:33:55.809149 22366 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:33:55.809178 22362 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:33:55.809152 22363 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:33:55.810096 22354 server_base.cc:1034] running on GCE node
I20260811 06:33:55.811026 22354 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:33:55.812706 22354 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:33:55.813970 22354 hybrid_clock.cc:648] HybridClock initialized: now 1786430035813896 us; error 67 us; skew 500 ppm
I20260811 06:33:55.814389 22354 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:33:55 dist-test-slave-gkw7 krb5kdc[21544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430035, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:33:55.821940 22354 init.cc:377] Logged in from keytab as kudu/127.28.239.254@KRBTEST.COM (short username kudu)
I20260811 06:33:55.823783 22354 webserver.cc:460] Webserver started at http://127.28.239.254:37769/ using document root <none> and password file <none>
I20260811 06:33:55.824559 22354 fs_manager.cc:359] Metadata directory not provided
I20260811 06:33:55.824669 22354 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:33:55.831466 22354 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 06:33:55.834841 22374 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:33:55.836390 22354 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 06:33:55.836602 22354 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/master-0/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/master-0/wal
uuid: "f9315cc436a5414ea115e916586d7497"
format_stamp: "Formatted at 2026-08-11 06:33:32 on dist-test-slave-gkw7"
server_key: "7f5f786b8c78515694591e9397b4c3e3"
server_key_iv: "198e4780229f0fb00f84ac5a2090c859"
server_key_version: "encryptionkey@0"
I20260811 06:33:55.837849 22354 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:33:55.869127 22354 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:33:55.874256 22354 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:33:55.874589 22354 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:33:55.895159 22354 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.254:42189
I20260811 06:33:55.895262 22443 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.254:42189 every 8 connection(s)
I20260811 06:33:55.896991 22354 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
I20260811 06:33:55.904999 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 22354
I20260811 06:33:55.905341 22447 sys_catalog.cc:263] Verifying existing consensus state
I20260811 06:33:55.906172 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.193:42485
--local_ip_for_outbound_sockets=127.28.239.193
--tserver_master_addrs=127.28.239.254:42189
--webserver_port=42879
--webserver_interface=127.28.239.193
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.193.keytab
--principal=kudu/127.28.239.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:43401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 06:33:55.910979 22447 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9315cc436a5414ea115e916586d7497: Bootstrap starting.
I20260811 06:33:55.934741 22447 log.cc:824] T 00000000000000000000000000000000 P f9315cc436a5414ea115e916586d7497: Log is configured to *not* fsync() on all Append() calls
I20260811 06:33:55.948035 22447 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9315cc436a5414ea115e916586d7497: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 06:33:55.948652 22447 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9315cc436a5414ea115e916586d7497: Bootstrap complete.
I20260811 06:33:55.958772 22447 raft_consensus.cc:348] T 00000000000000000000000000000000 P f9315cc436a5414ea115e916586d7497 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9315cc436a5414ea115e916586d7497" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 42189 } }
I20260811 06:33:55.959650 22447 raft_consensus.cc:729] T 00000000000000000000000000000000 P f9315cc436a5414ea115e916586d7497 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9315cc436a5414ea115e916586d7497, State: Initialized, Role: FOLLOWER
I20260811 06:33:55.960311 22447 consensus_queue.cc:260] T 00000000000000000000000000000000 P f9315cc436a5414ea115e916586d7497 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9315cc436a5414ea115e916586d7497" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 42189 } }
I20260811 06:33:55.960563 22447 raft_consensus.cc:388] T 00000000000000000000000000000000 P f9315cc436a5414ea115e916586d7497 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:33:55.960701 22447 raft_consensus.cc:482] T 00000000000000000000000000000000 P f9315cc436a5414ea115e916586d7497 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:33:55.960901 22447 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f9315cc436a5414ea115e916586d7497 [term 1 FOLLOWER]: Advancing to term 2
I20260811 06:33:55.965159 22447 raft_consensus.cc:504] T 00000000000000000000000000000000 P f9315cc436a5414ea115e916586d7497 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9315cc436a5414ea115e916586d7497" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 42189 } }
I20260811 06:33:55.965816 22447 leader_election.cc:302] T 00000000000000000000000000000000 P f9315cc436a5414ea115e916586d7497 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f9315cc436a5414ea115e916586d7497; no voters:
I20260811 06:33:55.966485 22447 leader_election.cc:288] T 00000000000000000000000000000000 P f9315cc436a5414ea115e916586d7497 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 06:33:55.967801 22452 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f9315cc436a5414ea115e916586d7497 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 06:33:55.970006 22452 raft_consensus.cc:686] T 00000000000000000000000000000000 P f9315cc436a5414ea115e916586d7497 [term 2 LEADER]: Becoming Leader. State: Replica: f9315cc436a5414ea115e916586d7497, State: Running, Role: LEADER
I20260811 06:33:55.970314 22447 sys_catalog.cc:564] T 00000000000000000000000000000000 P f9315cc436a5414ea115e916586d7497 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:33:55.970616 22452 consensus_queue.cc:237] T 00000000000000000000000000000000 P f9315cc436a5414ea115e916586d7497 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9315cc436a5414ea115e916586d7497" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 42189 } }
I20260811 06:33:55.975126 22454 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9315cc436a5414ea115e916586d7497 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f9315cc436a5414ea115e916586d7497. Latest consensus state: current_term: 2 leader_uuid: "f9315cc436a5414ea115e916586d7497" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9315cc436a5414ea115e916586d7497" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 42189 } } }
I20260811 06:33:55.975646 22454 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9315cc436a5414ea115e916586d7497 [sys.catalog]: This master's current role is: LEADER
I20260811 06:33:55.976316 22453 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9315cc436a5414ea115e916586d7497 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "f9315cc436a5414ea115e916586d7497" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9315cc436a5414ea115e916586d7497" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 42189 } } }
I20260811 06:33:55.976663 22453 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9315cc436a5414ea115e916586d7497 [sys.catalog]: This master's current role is: LEADER
I20260811 06:33:55.985030 22470 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:33:55.994117 22470 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:33:55.995738 22470 catalog_manager.cc:1259] Loaded cluster ID: 8ea31dfa1ea24bde8f419dac748e64f0
I20260811 06:33:55.995893 22470 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:33:55.999989 22470 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:33:56.001709 22470 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f9315cc436a5414ea115e916586d7497: Loaded TSK: 0
I20260811 06:33:56.032930 22470 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f9315cc436a5414ea115e916586d7497: Generated new TSK 1
I20260811 06:33:56.033406 22470 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 06:33:56.289075 22450 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:33:56.289381 22450 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:33:56.289582 22450 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:33:56.297024 22450 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:33:56.297144 22450 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:33:56.297330 22450 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.193
I20260811 06:33:56.307475 22450 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:43401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.193.keytab
--principal=kudu/127.28.239.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.193:42485
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.239.193
--webserver_port=42879
--tserver_master_addrs=127.28.239.254:42189
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.193
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:33:56.308903 22450 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:33:56.310359 22450 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:33:56.326905 22489 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:33:56.326906 22491 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:33:56.328297 22493 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:33:56.328601 22450 server_base.cc:1034] running on GCE node
I20260811 06:33:56.329380 22450 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:33:56.330464 22450 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:33:56.331703 22450 hybrid_clock.cc:648] HybridClock initialized: now 1786430036331628 us; error 67 us; skew 500 ppm
I20260811 06:33:56.332049 22450 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:33:56 dist-test-slave-gkw7 krb5kdc[21544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430036, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:33:56.335407 22450 init.cc:377] Logged in from keytab as kudu/127.28.239.193@KRBTEST.COM (short username kudu)
I20260811 06:33:56.336966 22450 webserver.cc:460] Webserver started at http://127.28.239.193:42879/ using document root <none> and password file <none>
I20260811 06:33:56.337787 22450 fs_manager.cc:359] Metadata directory not provided
I20260811 06:33:56.337955 22450 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:33:56.343855 22450 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.001s
I20260811 06:33:56.346958 22501 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:33:56.348528 22450 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 06:33:56.348719 22450 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-0/wal
uuid: "5e51526ceea347ba88b10bc1a50d84ca"
format_stamp: "Formatted at 2026-08-11 06:33:32 on dist-test-slave-gkw7"
server_key: "5cf5d515f041520e8b8894e158130968"
server_key_iv: "9bf9a52c50f3d66cdf0d1c57c1f9e85f"
server_key_version: "encryptionkey@0"
I20260811 06:33:56.349829 22450 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:33:56.371129 22450 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:33:56.375864 22450 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:33:56.376178 22450 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:33:56.377259 22450 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:33:56.378726 22450 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:33:56.378851 22450 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:33:56.378996 22450 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:33:56.379086 22450 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:33:56.414970 22450 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.193:42485
I20260811 06:33:56.415064 22679 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.193:42485 every 8 connection(s)
I20260811 06:33:56.416487 22450 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
I20260811 06:33:56.421680 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 22450
I20260811 06:33:56.422384 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.194:42877
--local_ip_for_outbound_sockets=127.28.239.194
--tserver_master_addrs=127.28.239.254:42189
--webserver_port=40013
--webserver_interface=127.28.239.194
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.194.keytab
--principal=kudu/127.28.239.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:43401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 06:33:56 dist-test-slave-gkw7 krb5kdc[21544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430036, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.193@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:33:56.444245 22685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:56.419567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.193:40983 (local address 127.28.239.254:42189)
0811 06:33:56.420161 (+ 594us) server_negotiation.cc:206] Beginning negotiation
0811 06:33:56.420174 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 06:33:56.420988 (+ 814us) server_negotiation.cc:407] Connection header received
0811 06:33:56.423582 (+ 2594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:33:56.423606 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:33:56.424130 (+ 524us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:33:56.424575 (+ 445us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:33:56.426343 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:56.428114 (+ 1771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:56.430214 (+ 2100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:56.430897 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:56.431115 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:56.434999 (+ 3884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:33:56.435007 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:33:56.435022 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:33:56.435065 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:33:56.439212 (+ 4147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:33:56.440028 (+ 816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:33:56.440037 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:33:56.440046 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:33:56.440225 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:33:56.440698 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:33:56.440703 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:33:56.440707 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:33:56.441272 (+ 565us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:33:56.441490 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0811 06:33:56.442416 (+ 926us) server_negotiation.cc:299] Negotiation successful
0811 06:33:56.442840 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":238,"threads_started":1}
I20260811 06:33:56.444245 22686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:56.419866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:42189 (local address 127.28.239.193:40983)
0811 06:33:56.420799 (+ 933us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:56.420846 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 06:33:56.422923 (+ 2077us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:33:56.424739 (+ 1816us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:33:56.424747 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:33:56.425339 (+ 592us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:33:56.426061 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:33:56.426135 (+ 74us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:33:56.428327 (+ 2192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:33:56.428334 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:33:56.430051 (+ 1717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:33:56.430070 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:33:56.431179 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:33:56.431195 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:33:56.431477 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:56.432206 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:33:56.432235 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:33:56.434697 (+ 2462us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:33:56.439376 (+ 4679us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:33:56.439386 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:33:56.439402 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:33:56.439867 (+ 465us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:33:56.440351 (+ 484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:33:56.440358 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:33:56.440361 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:33:56.440561 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:33:56.441472 (+ 911us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:33:56.441480 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:33:56.441872 (+ 392us) client_negotiation.cc:769] Sending connection context
0811 06:33:56.442240 (+ 368us) client_negotiation.cc:240] Negotiation successful
0811 06:33:56.442854 (+ 614us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":600,"thread_start_us":389,"threads_started":1}
I20260811 06:33:56.447324 22681 heartbeater.cc:347] Connected to a master server at 127.28.239.254:42189
I20260811 06:33:56.447716 22681 heartbeater.cc:464] Registering TS with master...
I20260811 06:33:56.448779 22681 heartbeater.cc:511] Master 127.28.239.254:42189 requested a full tablet report, sending...
I20260811 06:33:56.451296 22400 ts_manager.cc:194] Registered new tserver with Master: 5e51526ceea347ba88b10bc1a50d84ca (127.28.239.193:42485)
I20260811 06:33:56.452834 22400 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.239.193@KRBTEST.COM'} at 127.28.239.193:40983
W20260811 06:33:56.779610 22687 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:33:56.779903 22687 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:33:56.779999 22687 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:33:56.787180 22687 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:33:56.787292 22687 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:33:56.787407 22687 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.194
I20260811 06:33:56.797087 22687 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:43401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.194.keytab
--principal=kudu/127.28.239.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.194:42877
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.239.194
--webserver_port=40013
--tserver_master_addrs=127.28.239.254:42189
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.194
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:33:56.798357 22687 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:33:56.799641 22687 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:33:56.813146 22698 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:33:56.813233 22697 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:33:56.813233 22701 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:33:56.814217 22687 server_base.cc:1034] running on GCE node
I20260811 06:33:56.814869 22687 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:33:56.815836 22687 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:33:56.817078 22687 hybrid_clock.cc:648] HybridClock initialized: now 1786430036817030 us; error 60 us; skew 500 ppm
I20260811 06:33:56.817425 22687 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:33:56 dist-test-slave-gkw7 krb5kdc[21544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430036, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:33:56.820712 22687 init.cc:377] Logged in from keytab as kudu/127.28.239.194@KRBTEST.COM (short username kudu)
I20260811 06:33:56.822383 22687 webserver.cc:460] Webserver started at http://127.28.239.194:40013/ using document root <none> and password file <none>
I20260811 06:33:56.823103 22687 fs_manager.cc:359] Metadata directory not provided
I20260811 06:33:56.823267 22687 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:33:56.829007 22687 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 06:33:56.832290 22712 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:33:56.833875 22687 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 06:33:56.834088 22687 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-1/wal
uuid: "8c88b13171e84962a6de95aeeda55c2c"
format_stamp: "Formatted at 2026-08-11 06:33:33 on dist-test-slave-gkw7"
server_key: "49fbc2a75d3880e5bb58a29c326e0db9"
server_key_iv: "8e68b52c34c37c885ab9340e5c75e9cd"
server_key_version: "encryptionkey@0"
I20260811 06:33:56.835212 22687 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:33:56.851497 22687 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:33:56.856243 22687 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:33:56.856562 22687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:33:56.857724 22687 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:33:56.859216 22687 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:33:56.859357 22687 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:33:56.859498 22687 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:33:56.859613 22687 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:33:56.902057 22687 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.194:42877
I20260811 06:33:56.902140 22872 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.194:42877 every 8 connection(s)
I20260811 06:33:56.904039 22687 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
I20260811 06:33:56.908900 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 22687
I20260811 06:33:56.909861 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.195:45519
--local_ip_for_outbound_sockets=127.28.239.195
--tserver_master_addrs=127.28.239.254:42189
--webserver_port=41801
--webserver_interface=127.28.239.195
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.195.keytab
--principal=kudu/127.28.239.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:43401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 06:33:56 dist-test-slave-gkw7 krb5kdc[21544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430036, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.194@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:33:56.932117 22685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:56.907439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.194:48093 (local address 127.28.239.254:42189)
0811 06:33:56.907658 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0811 06:33:56.907663 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:33:56.909319 (+ 1656us) server_negotiation.cc:407] Connection header received
0811 06:33:56.912829 (+ 3510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:33:56.912836 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:33:56.912965 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:33:56.913082 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:33:56.915554 (+ 2472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:56.916539 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:56.919523 (+ 2984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:56.920010 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:56.920190 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:56.924750 (+ 4560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:33:56.924756 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:33:56.924760 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:33:56.924800 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:33:56.928008 (+ 3208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:33:56.929159 (+ 1151us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:33:56.929164 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:33:56.929166 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:33:56.929318 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:33:56.929962 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:33:56.929966 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:33:56.929968 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:33:56.930431 (+ 463us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:33:56.930557 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 06:33:56.931632 (+ 1075us) server_negotiation.cc:299] Negotiation successful
0811 06:33:56.931834 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 06:33:56.934070 22876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:56.907974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:42189 (local address 127.28.239.194:48093)
0811 06:33:56.909115 (+ 1141us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:56.909176 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 06:33:56.912422 (+ 3246us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:33:56.913405 (+ 983us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:33:56.913421 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:33:56.914374 (+ 953us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:33:56.915317 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:33:56.915349 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:33:56.916753 (+ 1404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:33:56.916762 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:33:56.919348 (+ 2586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:33:56.919368 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:33:56.920300 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:33:56.920310 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:33:56.920581 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:56.921450 (+ 869us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:33:56.921476 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:33:56.924411 (+ 2935us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:33:56.928185 (+ 3774us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:33:56.928199 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:33:56.928222 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:33:56.928949 (+ 727us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:33:56.929453 (+ 504us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:33:56.929462 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:33:56.929469 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:33:56.929822 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:33:56.930575 (+ 753us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:33:56.930599 (+ 24us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:33:56.931146 (+ 547us) client_negotiation.cc:769] Sending connection context
0811 06:33:56.931543 (+ 397us) client_negotiation.cc:240] Negotiation successful
0811 06:33:56.932283 (+ 740us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":707,"thread_start_us":405,"threads_started":1}
I20260811 06:33:56.935839 22874 heartbeater.cc:347] Connected to a master server at 127.28.239.254:42189
I20260811 06:33:56.936213 22874 heartbeater.cc:464] Registering TS with master...
I20260811 06:33:56.937068 22874 heartbeater.cc:511] Master 127.28.239.254:42189 requested a full tablet report, sending...
I20260811 06:33:56.938961 22400 ts_manager.cc:194] Registered new tserver with Master: 8c88b13171e84962a6de95aeeda55c2c (127.28.239.194:42877)
I20260811 06:33:56.940498 22400 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.239.194@KRBTEST.COM'} at 127.28.239.194:48093
W20260811 06:33:57.269769 22877 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:33:57.270067 22877 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:33:57.270268 22877 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:33:57.281909 22877 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:33:57.282069 22877 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:33:57.282279 22877 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.195
I20260811 06:33:57.292697 22877 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:43401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.195.keytab
--principal=kudu/127.28.239.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.195:45519
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.239.195
--webserver_port=41801
--tserver_master_addrs=127.28.239.254:42189
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.195
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:33:57.294171 22877 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:33:57.295519 22877 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:33:57.309832 22882 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:33:57.309790 22884 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:33:57.309748 22887 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:33:57.310791 22877 server_base.cc:1034] running on GCE node
I20260811 06:33:57.311475 22877 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:33:57.312538 22877 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:33:57.313795 22877 hybrid_clock.cc:648] HybridClock initialized: now 1786430037313742 us; error 46 us; skew 500 ppm
I20260811 06:33:57.314160 22877 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:33:57 dist-test-slave-gkw7 krb5kdc[21544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430037, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:33:57.318426 22877 init.cc:377] Logged in from keytab as kudu/127.28.239.195@KRBTEST.COM (short username kudu)
I20260811 06:33:57.320066 22877 webserver.cc:460] Webserver started at http://127.28.239.195:41801/ using document root <none> and password file <none>
I20260811 06:33:57.320858 22877 fs_manager.cc:359] Metadata directory not provided
I20260811 06:33:57.321033 22877 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:33:57.326898 22877 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 06:33:57.329831 22900 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:33:57.331358 22877 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 06:33:57.331552 22877 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-2/wal
uuid: "1648c41809414c8f97307d271b2541d6"
format_stamp: "Formatted at 2026-08-11 06:33:34 on dist-test-slave-gkw7"
server_key: "9091360401284f9d006d7621f18df1ff"
server_key_iv: "0647b0b07517332b949870b3d5b6b892"
server_key_version: "encryptionkey@0"
I20260811 06:33:57.332660 22877 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:33:57.346696 22877 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:33:57.351155 22877 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:33:57.351455 22877 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:33:57.352530 22877 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:33:57.354143 22877 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:33:57.354264 22877 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:33:57.354398 22877 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:33:57.354486 22877 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:33:57.394217 22877 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.195:45519
I20260811 06:33:57.394316 23088 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.195:45519 every 8 connection(s)
I20260811 06:33:57.396116 22877 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
I20260811 06:33:57.399825 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 22877
Aug 11 06:33:57 dist-test-slave-gkw7 krb5kdc[21544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430037, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.195@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:33:57.417698 23092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:57.401019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38650 (local address 127.28.239.254:42189)
0811 06:33:57.401584 (+ 565us) server_negotiation.cc:206] Beginning negotiation
0811 06:33:57.401590 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:33:57.401915 (+ 325us) server_negotiation.cc:407] Connection header received
0811 06:33:57.402148 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:33:57.402153 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:33:57.402222 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:33:57.402344 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:33:57.403977 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:57.404761 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:57.406303 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:57.406793 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:57.406933 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:57.410141 (+ 3208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:33:57.410150 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:33:57.410158 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:33:57.410206 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:33:57.414001 (+ 3795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:33:57.415011 (+ 1010us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:33:57.415017 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:33:57.415021 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:33:57.415216 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:33:57.415725 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:33:57.415733 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:33:57.415740 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:33:57.416115 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:33:57.416493 (+ 378us) server_negotiation.cc:1035] Waiting for connection context
0811 06:33:57.416841 (+ 348us) server_negotiation.cc:299] Negotiation successful
0811 06:33:57.417163 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":409,"thread_start_us":272,"threads_started":1}
I20260811 06:33:57.421849 22685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:57.399065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.195:54645 (local address 127.28.239.254:42189)
0811 06:33:57.399323 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0811 06:33:57.399328 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:33:57.400910 (+ 1582us) server_negotiation.cc:407] Connection header received
0811 06:33:57.403332 (+ 2422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:33:57.403338 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:33:57.403425 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:33:57.403558 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:33:57.405235 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:57.406433 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:57.408440 (+ 2007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:57.408900 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:57.409057 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:57.413503 (+ 4446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:33:57.413510 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:33:57.413516 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:33:57.413614 (+ 98us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:33:57.417478 (+ 3864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:33:57.418646 (+ 1168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:33:57.418651 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:33:57.418655 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:33:57.418825 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:33:57.419415 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:33:57.419420 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:33:57.419424 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:33:57.419759 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:33:57.419897 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 06:33:57.421121 (+ 1224us) server_negotiation.cc:299] Negotiation successful
0811 06:33:57.421389 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":64,"server-negotiator.queue_time_us":114}
I20260811 06:33:57.422967 23091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:57.399659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:42189 (local address 127.28.239.195:54645)
0811 06:33:57.400702 (+ 1043us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:57.400781 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0811 06:33:57.403031 (+ 2250us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:33:57.403841 (+ 810us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:33:57.403850 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:33:57.404418 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:33:57.405047 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:33:57.405066 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:33:57.406634 (+ 1568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:33:57.406639 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:33:57.408266 (+ 1627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:33:57.408280 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:33:57.409179 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:33:57.409188 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:33:57.409455 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:57.410249 (+ 794us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:33:57.410288 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:33:57.413261 (+ 2973us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:33:57.417696 (+ 4435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:33:57.417706 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:33:57.417723 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:33:57.418338 (+ 615us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:33:57.418971 (+ 633us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:33:57.418976 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:33:57.418980 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:33:57.419278 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:33:57.419911 (+ 633us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:33:57.419920 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:33:57.420397 (+ 477us) client_negotiation.cc:769] Sending connection context
0811 06:33:57.420964 (+ 567us) client_negotiation.cc:240] Negotiation successful
0811 06:33:57.421707 (+ 743us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":630,"thread_start_us":377,"threads_started":1}
I20260811 06:33:57.424631 23089 heartbeater.cc:347] Connected to a master server at 127.28.239.254:42189
I20260811 06:33:57.424978 23089 heartbeater.cc:464] Registering TS with master...
I20260811 06:33:57.425827 23089 heartbeater.cc:511] Master 127.28.239.254:42189 requested a full tablet report, sending...
I20260811 06:33:57.427403 22400 ts_manager.cc:194] Registered new tserver with Master: 1648c41809414c8f97307d271b2541d6 (127.28.239.195:45519)
I20260811 06:33:57.428275 22400 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.239.195@KRBTEST.COM'} at 127.28.239.195:54645
I20260811 06:33:57.433387 29631 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 06:33:57.440773 22685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:57.434422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38658 (local address 127.28.239.254:42189)
0811 06:33:57.434595 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 06:33:57.434600 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:33:57.435273 (+ 673us) server_negotiation.cc:407] Connection header received
0811 06:33:57.435515 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:33:57.435518 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:33:57.435670 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:33:57.435794 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:33:57.436306 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:57.437089 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:57.438828 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:57.439250 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:57.439344 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:57.439910 (+ 566us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:33:57.440266 (+ 356us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0811 06:33:57.440572 (+ 306us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:38658: token expired
Metrics: {"server-negotiator.queue_time_us":64}
W20260811 06:33:57.440939 22685 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:38658: token expired
W20260811 06:33:57.441658 22293 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.28.239.254:42189)
I20260811 06:33:57.455899 22685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:57.442419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38660 (local address 127.28.239.254:42189)
0811 06:33:57.442607 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 06:33:57.442611 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:33:57.442626 (+ 15us) server_negotiation.cc:407] Connection header received
0811 06:33:57.442663 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:33:57.442667 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:33:57.442720 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:33:57.442823 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:33:57.444192 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:57.444930 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:57.446457 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:57.446871 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:57.447006 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:57.449874 (+ 2868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:33:57.449879 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:33:57.449882 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:33:57.449917 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:33:57.452685 (+ 2768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:33:57.453665 (+ 980us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:33:57.453671 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:33:57.453676 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:33:57.453809 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:33:57.454255 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:33:57.454259 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:33:57.454261 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:33:57.454667 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:33:57.454939 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0811 06:33:57.455293 (+ 354us) server_negotiation.cc:299] Negotiation successful
0811 06:33:57.455532 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 06:33:57.457226 22681 heartbeater.cc:503] Master 127.28.239.254:42189 was elected leader, sending a full tablet report...
I20260811 06:33:57.460305 22399 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38660:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:33:57.469261 22399 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:33:57.493650 23105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:57.482504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46990 (local address 127.28.239.193:42485)
0811 06:33:57.483360 (+ 856us) server_negotiation.cc:206] Beginning negotiation
0811 06:33:57.483374 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 06:33:57.484046 (+ 672us) server_negotiation.cc:407] Connection header received
0811 06:33:57.484376 (+ 330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:33:57.484386 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:33:57.484655 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:33:57.484833 (+ 178us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:33:57.485928 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:57.487634 (+ 1706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:57.490620 (+ 2986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:57.492265 (+ 1645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:57.492474 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:57.492662 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0811 06:33:57.493056 (+ 394us) server_negotiation.cc:299] Negotiation successful
0811 06:33:57.493264 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":615,"thread_start_us":400,"threads_started":1}
I20260811 06:33:57.493690 23101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:57.482764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.195:45519 (local address 127.0.0.1:51434)
0811 06:33:57.483915 (+ 1151us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:57.483941 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 06:33:57.484116 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:33:57.486335 (+ 2219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:33:57.486340 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:33:57.486374 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:33:57.486769 (+ 395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:33:57.486782 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:33:57.488710 (+ 1928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:33:57.488716 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:33:57.491132 (+ 2416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:33:57.491149 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:33:57.492801 (+ 1652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:33:57.492807 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:33:57.492970 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:57.493002 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 06:33:57.493055 (+ 53us) client_negotiation.cc:240] Negotiation successful
0811 06:33:57.493200 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1029,"mutex_wait_us":117,"thread_start_us":840,"threads_started":1}
I20260811 06:33:57.494297 23108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:57.482352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.193:42485 (local address 127.0.0.1:46990)
0811 06:33:57.483888 (+ 1536us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:57.483941 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 06:33:57.484168 (+ 227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:33:57.485185 (+ 1017us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:33:57.485194 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:33:57.485231 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:33:57.485706 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:33:57.485722 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:33:57.487813 (+ 2091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:33:57.487818 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:33:57.490435 (+ 2617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:33:57.490454 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:33:57.492516 (+ 2062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:33:57.492523 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:33:57.492785 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:57.492846 (+ 61us) client_negotiation.cc:769] Sending connection context
0811 06:33:57.492983 (+ 137us) client_negotiation.cc:240] Negotiation successful
0811 06:33:57.493187 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1259,"mutex_wait_us":5,"thread_start_us":1263,"threads_started":1}
I20260811 06:33:57.494805 23109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:57.483444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51434 (local address 127.28.239.195:45519)
0811 06:33:57.485659 (+ 2215us) server_negotiation.cc:206] Beginning negotiation
0811 06:33:57.485668 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:33:57.485704 (+ 36us) server_negotiation.cc:407] Connection header received
0811 06:33:57.485801 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:33:57.485822 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:33:57.486086 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:33:57.486417 (+ 331us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:33:57.486954 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:57.488548 (+ 1594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:57.491309 (+ 2761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:57.492551 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:57.493394 (+ 843us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:57.493582 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0811 06:33:57.493667 (+ 85us) server_negotiation.cc:299] Negotiation successful
0811 06:33:57.494548 (+ 881us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1990,"thread_start_us":1813,"threads_started":1}
I20260811 06:33:57.494858 23113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:57.484472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.194:42877 (local address 127.0.0.1:52582)
0811 06:33:57.486472 (+ 2000us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:57.486498 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 06:33:57.486647 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:33:57.487559 (+ 912us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:33:57.487566 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:33:57.487592 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:33:57.488036 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:33:57.488048 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:33:57.489566 (+ 1518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:33:57.489571 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:33:57.491542 (+ 1971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:33:57.491558 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:33:57.493664 (+ 2106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:33:57.493678 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:33:57.494182 (+ 504us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:57.494207 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 06:33:57.494329 (+ 122us) client_negotiation.cc:240] Negotiation successful
0811 06:33:57.494465 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1851,"mutex_wait_us":23,"thread_start_us":1697,"threads_started":1}
I20260811 06:33:57.495885 23116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:57.485098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52582 (local address 127.28.239.194:42877)
0811 06:33:57.485968 (+ 870us) server_negotiation.cc:206] Beginning negotiation
0811 06:33:57.485980 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 06:33:57.486732 (+ 752us) server_negotiation.cc:407] Connection header received
0811 06:33:57.486840 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:33:57.486846 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:33:57.487103 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:33:57.487316 (+ 213us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:33:57.488245 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:57.489393 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:57.491751 (+ 2358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:57.493112 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:57.493278 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:57.493439 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 06:33:57.494373 (+ 934us) server_negotiation.cc:299] Negotiation successful
0811 06:33:57.495527 (+ 1154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":678,"thread_start_us":519,"threads_started":1}
I20260811 06:33:57.498777 22565 tablet_service.cc:1467] Processing CreateTablet for tablet 205353c8754145da9bbc49b9dd2ac784 (DEFAULT_TABLE table=test-table [id=fa13dd6dd8ac432c963022ab3d38fd96]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:33:57.498790 22978 tablet_service.cc:1467] Processing CreateTablet for tablet 205353c8754145da9bbc49b9dd2ac784 (DEFAULT_TABLE table=test-table [id=fa13dd6dd8ac432c963022ab3d38fd96]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:33:57.498994 22779 tablet_service.cc:1467] Processing CreateTablet for tablet 205353c8754145da9bbc49b9dd2ac784 (DEFAULT_TABLE table=test-table [id=fa13dd6dd8ac432c963022ab3d38fd96]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:33:57.500304 22978 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 205353c8754145da9bbc49b9dd2ac784. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:33:57.500314 22565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 205353c8754145da9bbc49b9dd2ac784. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:33:57.500314 22779 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 205353c8754145da9bbc49b9dd2ac784. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:33:57.514755 23118 tablet_bootstrap.cc:492] T 205353c8754145da9bbc49b9dd2ac784 P 8c88b13171e84962a6de95aeeda55c2c: Bootstrap starting.
I20260811 06:33:57.516305 23119 tablet_bootstrap.cc:492] T 205353c8754145da9bbc49b9dd2ac784 P 1648c41809414c8f97307d271b2541d6: Bootstrap starting.
I20260811 06:33:57.517163 23120 tablet_bootstrap.cc:492] T 205353c8754145da9bbc49b9dd2ac784 P 5e51526ceea347ba88b10bc1a50d84ca: Bootstrap starting.
I20260811 06:33:57.519030 23118 tablet_bootstrap.cc:654] T 205353c8754145da9bbc49b9dd2ac784 P 8c88b13171e84962a6de95aeeda55c2c: Neither blocks nor log segments found. Creating new log.
I20260811 06:33:57.520052 23119 tablet_bootstrap.cc:654] T 205353c8754145da9bbc49b9dd2ac784 P 1648c41809414c8f97307d271b2541d6: Neither blocks nor log segments found. Creating new log.
I20260811 06:33:57.520450 23118 log.cc:824] T 205353c8754145da9bbc49b9dd2ac784 P 8c88b13171e84962a6de95aeeda55c2c: Log is configured to *not* fsync() on all Append() calls
I20260811 06:33:57.520959 23120 tablet_bootstrap.cc:654] T 205353c8754145da9bbc49b9dd2ac784 P 5e51526ceea347ba88b10bc1a50d84ca: Neither blocks nor log segments found. Creating new log.
I20260811 06:33:57.521257 23119 log.cc:824] T 205353c8754145da9bbc49b9dd2ac784 P 1648c41809414c8f97307d271b2541d6: Log is configured to *not* fsync() on all Append() calls
I20260811 06:33:57.522327 23120 log.cc:824] T 205353c8754145da9bbc49b9dd2ac784 P 5e51526ceea347ba88b10bc1a50d84ca: Log is configured to *not* fsync() on all Append() calls
I20260811 06:33:57.524590 23118 tablet_bootstrap.cc:492] T 205353c8754145da9bbc49b9dd2ac784 P 8c88b13171e84962a6de95aeeda55c2c: No bootstrap required, opened a new log
I20260811 06:33:57.524885 23118 ts_tablet_manager.cc:1397] T 205353c8754145da9bbc49b9dd2ac784 P 8c88b13171e84962a6de95aeeda55c2c: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260811 06:33:57.525833 23120 tablet_bootstrap.cc:492] T 205353c8754145da9bbc49b9dd2ac784 P 5e51526ceea347ba88b10bc1a50d84ca: No bootstrap required, opened a new log
I20260811 06:33:57.526132 23120 ts_tablet_manager.cc:1397] T 205353c8754145da9bbc49b9dd2ac784 P 5e51526ceea347ba88b10bc1a50d84ca: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260811 06:33:57.526889 23119 tablet_bootstrap.cc:492] T 205353c8754145da9bbc49b9dd2ac784 P 1648c41809414c8f97307d271b2541d6: No bootstrap required, opened a new log
I20260811 06:33:57.527145 23119 ts_tablet_manager.cc:1397] T 205353c8754145da9bbc49b9dd2ac784 P 1648c41809414c8f97307d271b2541d6: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260811 06:33:57.533871 23120 raft_consensus.cc:348] T 205353c8754145da9bbc49b9dd2ac784 P 5e51526ceea347ba88b10bc1a50d84ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e51526ceea347ba88b10bc1a50d84ca" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 42485 } } peers { permanent_uuid: "8c88b13171e84962a6de95aeeda55c2c" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 42877 } } peers { permanent_uuid: "1648c41809414c8f97307d271b2541d6" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 45519 } }
I20260811 06:33:57.534441 23120 raft_consensus.cc:374] T 205353c8754145da9bbc49b9dd2ac784 P 5e51526ceea347ba88b10bc1a50d84ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:33:57.534574 23120 raft_consensus.cc:729] T 205353c8754145da9bbc49b9dd2ac784 P 5e51526ceea347ba88b10bc1a50d84ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e51526ceea347ba88b10bc1a50d84ca, State: Initialized, Role: FOLLOWER
I20260811 06:33:57.535239 23120 consensus_queue.cc:260] T 205353c8754145da9bbc49b9dd2ac784 P 5e51526ceea347ba88b10bc1a50d84ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e51526ceea347ba88b10bc1a50d84ca" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 42485 } } peers { permanent_uuid: "8c88b13171e84962a6de95aeeda55c2c" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 42877 } } peers { permanent_uuid: "1648c41809414c8f97307d271b2541d6" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 45519 } }
I20260811 06:33:57.536007 23118 raft_consensus.cc:348] T 205353c8754145da9bbc49b9dd2ac784 P 8c88b13171e84962a6de95aeeda55c2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e51526ceea347ba88b10bc1a50d84ca" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 42485 } } peers { permanent_uuid: "8c88b13171e84962a6de95aeeda55c2c" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 42877 } } peers { permanent_uuid: "1648c41809414c8f97307d271b2541d6" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 45519 } }
I20260811 06:33:57.536460 23118 raft_consensus.cc:374] T 205353c8754145da9bbc49b9dd2ac784 P 8c88b13171e84962a6de95aeeda55c2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:33:57.536553 23118 raft_consensus.cc:729] T 205353c8754145da9bbc49b9dd2ac784 P 8c88b13171e84962a6de95aeeda55c2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c88b13171e84962a6de95aeeda55c2c, State: Initialized, Role: FOLLOWER
I20260811 06:33:57.537114 23120 ts_tablet_manager.cc:1428] T 205353c8754145da9bbc49b9dd2ac784 P 5e51526ceea347ba88b10bc1a50d84ca: Time spent starting tablet: real 0.011s user 0.010s sys 0.000s
I20260811 06:33:57.537256 23118 consensus_queue.cc:260] T 205353c8754145da9bbc49b9dd2ac784 P 8c88b13171e84962a6de95aeeda55c2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e51526ceea347ba88b10bc1a50d84ca" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 42485 } } peers { permanent_uuid: "8c88b13171e84962a6de95aeeda55c2c" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 42877 } } peers { permanent_uuid: "1648c41809414c8f97307d271b2541d6" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 45519 } }
I20260811 06:33:57.537988 23119 raft_consensus.cc:348] T 205353c8754145da9bbc49b9dd2ac784 P 1648c41809414c8f97307d271b2541d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e51526ceea347ba88b10bc1a50d84ca" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 42485 } } peers { permanent_uuid: "8c88b13171e84962a6de95aeeda55c2c" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 42877 } } peers { permanent_uuid: "1648c41809414c8f97307d271b2541d6" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 45519 } }
I20260811 06:33:57.538446 23119 raft_consensus.cc:374] T 205353c8754145da9bbc49b9dd2ac784 P 1648c41809414c8f97307d271b2541d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:33:57.538475 22874 heartbeater.cc:503] Master 127.28.239.254:42189 was elected leader, sending a full tablet report...
I20260811 06:33:57.538597 23119 raft_consensus.cc:729] T 205353c8754145da9bbc49b9dd2ac784 P 1648c41809414c8f97307d271b2541d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1648c41809414c8f97307d271b2541d6, State: Initialized, Role: FOLLOWER
I20260811 06:33:57.539218 23118 ts_tablet_manager.cc:1428] T 205353c8754145da9bbc49b9dd2ac784 P 8c88b13171e84962a6de95aeeda55c2c: Time spent starting tablet: real 0.014s user 0.002s sys 0.014s
I20260811 06:33:57.539273 23119 consensus_queue.cc:260] T 205353c8754145da9bbc49b9dd2ac784 P 1648c41809414c8f97307d271b2541d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e51526ceea347ba88b10bc1a50d84ca" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 42485 } } peers { permanent_uuid: "8c88b13171e84962a6de95aeeda55c2c" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 42877 } } peers { permanent_uuid: "1648c41809414c8f97307d271b2541d6" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 45519 } }
I20260811 06:33:57.541833 23089 heartbeater.cc:503] Master 127.28.239.254:42189 was elected leader, sending a full tablet report...
I20260811 06:33:57.542421 23119 ts_tablet_manager.cc:1428] T 205353c8754145da9bbc49b9dd2ac784 P 1648c41809414c8f97307d271b2541d6: Time spent starting tablet: real 0.015s user 0.016s sys 0.000s
I20260811 06:33:57.595012 23128 raft_consensus.cc:482] T 205353c8754145da9bbc49b9dd2ac784 P 5e51526ceea347ba88b10bc1a50d84ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:33:57.595337 23128 raft_consensus.cc:504] T 205353c8754145da9bbc49b9dd2ac784 P 5e51526ceea347ba88b10bc1a50d84ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e51526ceea347ba88b10bc1a50d84ca" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 42485 } } peers { permanent_uuid: "8c88b13171e84962a6de95aeeda55c2c" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 42877 } } peers { permanent_uuid: "1648c41809414c8f97307d271b2541d6" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 45519 } }
I20260811 06:33:57.597203 23128 leader_election.cc:288] T 205353c8754145da9bbc49b9dd2ac784 P 5e51526ceea347ba88b10bc1a50d84ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c88b13171e84962a6de95aeeda55c2c (127.28.239.194:42877), 1648c41809414c8f97307d271b2541d6 (127.28.239.195:45519)
I20260811 06:33:57.604453 23116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:57.597618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.193:55937 (local address 127.28.239.194:42877)
0811 06:33:57.597895 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0811 06:33:57.597904 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:33:57.598772 (+ 868us) server_negotiation.cc:407] Connection header received
0811 06:33:57.598949 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:33:57.598955 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:33:57.599039 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:33:57.599138 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:33:57.599811 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:57.600621 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:57.602625 (+ 2004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:57.603386 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:57.603507 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:57.603598 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 06:33:57.604032 (+ 434us) server_negotiation.cc:299] Negotiation successful
0811 06:33:57.604174 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260811 06:33:57.604581 23131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:57.597354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.194:42877 (local address 127.28.239.193:55937)
0811 06:33:57.598644 (+ 1290us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:57.598664 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 06:33:57.598795 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:33:57.599176 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:33:57.599181 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:33:57.599209 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:33:57.599645 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:33:57.599657 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:33:57.600784 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:33:57.600789 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:33:57.602454 (+ 1665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:33:57.602467 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:33:57.603624 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:33:57.603632 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:33:57.603819 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:57.603888 (+ 69us) client_negotiation.cc:769] Sending connection context
0811 06:33:57.603962 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 06:33:57.604095 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1163,"mutex_wait_us":20,"thread_start_us":905,"threads_started":1}
I20260811 06:33:57.604632 23134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:57.597662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.195:45519 (local address 127.28.239.193:43973)
0811 06:33:57.598813 (+ 1151us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:57.598831 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 06:33:57.598994 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:33:57.599405 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:33:57.599409 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:33:57.599430 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:33:57.599866 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:33:57.599881 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:33:57.600951 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:33:57.600955 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:33:57.602459 (+ 1504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:33:57.602472 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:33:57.603717 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:33:57.603722 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:33:57.603916 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:57.603961 (+ 45us) client_negotiation.cc:769] Sending connection context
0811 06:33:57.604060 (+ 99us) client_negotiation.cc:240] Negotiation successful
0811 06:33:57.604190 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1000,"mutex_wait_us":46,"thread_start_us":679,"threads_started":1}
I20260811 06:33:57.604511 23109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:57.597833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.193:43973 (local address 127.28.239.195:45519)
0811 06:33:57.598083 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0811 06:33:57.598093 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:33:57.598957 (+ 864us) server_negotiation.cc:407] Connection header received
0811 06:33:57.599181 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:33:57.599187 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:33:57.599268 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:33:57.599373 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:33:57.600038 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:57.600803 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:57.602666 (+ 1863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:57.603479 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:57.603647 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:57.603721 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 06:33:57.604119 (+ 398us) server_negotiation.cc:299] Negotiation successful
0811 06:33:57.604270 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 06:33:57.605747 22810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "205353c8754145da9bbc49b9dd2ac784" candidate_uuid: "5e51526ceea347ba88b10bc1a50d84ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c88b13171e84962a6de95aeeda55c2c" is_pre_election: true
I20260811 06:33:57.606082 23015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "205353c8754145da9bbc49b9dd2ac784" candidate_uuid: "5e51526ceea347ba88b10bc1a50d84ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1648c41809414c8f97307d271b2541d6" is_pre_election: true
I20260811 06:33:57.606320 22810 raft_consensus.cc:2454] T 205353c8754145da9bbc49b9dd2ac784 P 8c88b13171e84962a6de95aeeda55c2c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e51526ceea347ba88b10bc1a50d84ca in term 0.
I20260811 06:33:57.606706 23015 raft_consensus.cc:2454] T 205353c8754145da9bbc49b9dd2ac784 P 1648c41809414c8f97307d271b2541d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e51526ceea347ba88b10bc1a50d84ca in term 0.
I20260811 06:33:57.607105 22505 leader_election.cc:302] T 205353c8754145da9bbc49b9dd2ac784 P 5e51526ceea347ba88b10bc1a50d84ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e51526ceea347ba88b10bc1a50d84ca, 8c88b13171e84962a6de95aeeda55c2c; no voters:
I20260811 06:33:57.607544 23128 raft_consensus.cc:2781] T 205353c8754145da9bbc49b9dd2ac784 P 5e51526ceea347ba88b10bc1a50d84ca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:33:57.607728 23128 raft_consensus.cc:482] T 205353c8754145da9bbc49b9dd2ac784 P 5e51526ceea347ba88b10bc1a50d84ca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:33:57.607851 23128 raft_consensus.cc:3037] T 205353c8754145da9bbc49b9dd2ac784 P 5e51526ceea347ba88b10bc1a50d84ca [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:33:57.610170 23128 raft_consensus.cc:504] T 205353c8754145da9bbc49b9dd2ac784 P 5e51526ceea347ba88b10bc1a50d84ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e51526ceea347ba88b10bc1a50d84ca" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 42485 } } peers { permanent_uuid: "8c88b13171e84962a6de95aeeda55c2c" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 42877 } } peers { permanent_uuid: "1648c41809414c8f97307d271b2541d6" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 45519 } }
I20260811 06:33:57.610760 23128 leader_election.cc:288] T 205353c8754145da9bbc49b9dd2ac784 P 5e51526ceea347ba88b10bc1a50d84ca [CANDIDATE]: Term 1 election: Requested vote from peers 8c88b13171e84962a6de95aeeda55c2c (127.28.239.194:42877), 1648c41809414c8f97307d271b2541d6 (127.28.239.195:45519)
I20260811 06:33:57.611291 22810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "205353c8754145da9bbc49b9dd2ac784" candidate_uuid: "5e51526ceea347ba88b10bc1a50d84ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c88b13171e84962a6de95aeeda55c2c"
I20260811 06:33:57.611279 23015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "205353c8754145da9bbc49b9dd2ac784" candidate_uuid: "5e51526ceea347ba88b10bc1a50d84ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1648c41809414c8f97307d271b2541d6"
I20260811 06:33:57.611495 22810 raft_consensus.cc:3037] T 205353c8754145da9bbc49b9dd2ac784 P 8c88b13171e84962a6de95aeeda55c2c [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:33:57.611496 23015 raft_consensus.cc:3037] T 205353c8754145da9bbc49b9dd2ac784 P 1648c41809414c8f97307d271b2541d6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:33:57.614856 22810 raft_consensus.cc:2454] T 205353c8754145da9bbc49b9dd2ac784 P 8c88b13171e84962a6de95aeeda55c2c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e51526ceea347ba88b10bc1a50d84ca in term 1.
I20260811 06:33:57.614856 23015 raft_consensus.cc:2454] T 205353c8754145da9bbc49b9dd2ac784 P 1648c41809414c8f97307d271b2541d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e51526ceea347ba88b10bc1a50d84ca in term 1.
I20260811 06:33:57.615321 22502 leader_election.cc:302] T 205353c8754145da9bbc49b9dd2ac784 P 5e51526ceea347ba88b10bc1a50d84ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1648c41809414c8f97307d271b2541d6, 5e51526ceea347ba88b10bc1a50d84ca, 8c88b13171e84962a6de95aeeda55c2c; no voters:
I20260811 06:33:57.615639 23128 raft_consensus.cc:2781] T 205353c8754145da9bbc49b9dd2ac784 P 5e51526ceea347ba88b10bc1a50d84ca [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:33:57.616161 23128 raft_consensus.cc:686] T 205353c8754145da9bbc49b9dd2ac784 P 5e51526ceea347ba88b10bc1a50d84ca [term 1 LEADER]: Becoming Leader. State: Replica: 5e51526ceea347ba88b10bc1a50d84ca, State: Running, Role: LEADER
I20260811 06:33:57.616731 23128 consensus_queue.cc:237] T 205353c8754145da9bbc49b9dd2ac784 P 5e51526ceea347ba88b10bc1a50d84ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e51526ceea347ba88b10bc1a50d84ca" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 42485 } } peers { permanent_uuid: "8c88b13171e84962a6de95aeeda55c2c" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 42877 } } peers { permanent_uuid: "1648c41809414c8f97307d271b2541d6" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 45519 } }
I20260811 06:33:57.621376 22399 catalog_manager.cc:5495] T 205353c8754145da9bbc49b9dd2ac784 P 5e51526ceea347ba88b10bc1a50d84ca reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e51526ceea347ba88b10bc1a50d84ca (127.28.239.193). New cstate: current_term: 1 leader_uuid: "5e51526ceea347ba88b10bc1a50d84ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e51526ceea347ba88b10bc1a50d84ca" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 42485 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c88b13171e84962a6de95aeeda55c2c" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 42877 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1648c41809414c8f97307d271b2541d6" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 45519 } health_report { overall_health: UNKNOWN } } }
W20260811 06:33:57.647975 23090 tablet.cc:2369] T 205353c8754145da9bbc49b9dd2ac784 P 1648c41809414c8f97307d271b2541d6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:33:57.650887 23105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:57.645091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47006 (local address 127.28.239.193:42485)
0811 06:33:57.645327 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0811 06:33:57.645332 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:33:57.645350 (+ 18us) server_negotiation.cc:407] Connection header received
0811 06:33:57.645443 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:33:57.645447 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:33:57.645562 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:33:57.645692 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:33:57.646198 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:57.646940 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:57.648477 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:57.649057 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:57.649165 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:57.649759 (+ 594us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:33:57.650293 (+ 534us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:33:57.650409 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 06:33:57.650527 (+ 118us) server_negotiation.cc:299] Negotiation successful
0811 06:33:57.650595 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
W20260811 06:33:57.656428 22875 tablet.cc:2369] T 205353c8754145da9bbc49b9dd2ac784 P 8c88b13171e84962a6de95aeeda55c2c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:33:57.659837 22810 raft_consensus.cc:1261] T 205353c8754145da9bbc49b9dd2ac784 P 8c88b13171e84962a6de95aeeda55c2c [term 1 FOLLOWER]: Refusing update from remote peer 5e51526ceea347ba88b10bc1a50d84ca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:33:57.659838 23015 raft_consensus.cc:1261] T 205353c8754145da9bbc49b9dd2ac784 P 1648c41809414c8f97307d271b2541d6 [term 1 FOLLOWER]: Refusing update from remote peer 5e51526ceea347ba88b10bc1a50d84ca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:33:57.660888 23128 consensus_queue.cc:1035] T 205353c8754145da9bbc49b9dd2ac784 P 5e51526ceea347ba88b10bc1a50d84ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "1648c41809414c8f97307d271b2541d6" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 45519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:33:57.661207 23135 consensus_queue.cc:1035] T 205353c8754145da9bbc49b9dd2ac784 P 5e51526ceea347ba88b10bc1a50d84ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c88b13171e84962a6de95aeeda55c2c" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 42877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 06:33:57.669180 22683 tablet.cc:2369] T 205353c8754145da9bbc49b9dd2ac784 P 5e51526ceea347ba88b10bc1a50d84ca: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:33:57.675140 23140 mvcc.cc:205] Tried to move back new op lower bound from 7317217434244116480 to 7317217434082590720. Current Snapshot: MvccSnapshot[applied={T|T < 7317217434244116480}]
I20260811 06:33:57.690045 22399 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38660:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:33:57.690336 22399 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38660:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:33:57.693775 22399 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f9315cc436a5414ea115e916586d7497: Sending DeleteTablet for 3 replicas of tablet 205353c8754145da9bbc49b9dd2ac784
I20260811 06:33:57.694969 22565 tablet_service.cc:1514] Processing DeleteTablet for tablet 205353c8754145da9bbc49b9dd2ac784 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:33:57 UTC) from {username='kudu', principal='kudu/127.28.239.254@KRBTEST.COM'} at 127.0.0.1:46990
I20260811 06:33:57.695083 22779 tablet_service.cc:1514] Processing DeleteTablet for tablet 205353c8754145da9bbc49b9dd2ac784 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:33:57 UTC) from {username='kudu', principal='kudu/127.28.239.254@KRBTEST.COM'} at 127.0.0.1:52582
I20260811 06:33:57.695410 22978 tablet_service.cc:1514] Processing DeleteTablet for tablet 205353c8754145da9bbc49b9dd2ac784 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:33:57 UTC) from {username='kudu', principal='kudu/127.28.239.254@KRBTEST.COM'} at 127.0.0.1:51434
I20260811 06:33:57.695827 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 22450
I20260811 06:33:57.708132 23151 tablet_replica.cc:330] T 205353c8754145da9bbc49b9dd2ac784 P 8c88b13171e84962a6de95aeeda55c2c: stopping tablet replica
I20260811 06:33:57.708899 23151 raft_consensus.cc:2229] T 205353c8754145da9bbc49b9dd2ac784 P 8c88b13171e84962a6de95aeeda55c2c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:33:57.709679 23151 raft_consensus.cc:2258] T 205353c8754145da9bbc49b9dd2ac784 P 8c88b13171e84962a6de95aeeda55c2c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:33:57.712466 23151 ts_tablet_manager.cc:1905] T 205353c8754145da9bbc49b9dd2ac784 P 8c88b13171e84962a6de95aeeda55c2c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:33:57.713220 23152 tablet_replica.cc:330] T 205353c8754145da9bbc49b9dd2ac784 P 5e51526ceea347ba88b10bc1a50d84ca: stopping tablet replica
I20260811 06:33:57.713788 23152 raft_consensus.cc:2229] T 205353c8754145da9bbc49b9dd2ac784 P 5e51526ceea347ba88b10bc1a50d84ca [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:33:57.721380 23151 ts_tablet_manager.cc:1918] T 205353c8754145da9bbc49b9dd2ac784 P 8c88b13171e84962a6de95aeeda55c2c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:33:57.721920 23105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:57.696799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47020 (local address 127.28.239.193:42485)
0811 06:33:57.696979 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 06:33:57.696986 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:33:57.697472 (+ 486us) server_negotiation.cc:407] Connection header received
0811 06:33:57.697589 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:33:57.697597 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:33:57.697709 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:33:57.697825 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:33:57.700351 (+ 2526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:57.702044 (+ 1693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:57.704636 (+ 2592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:57.705667 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:57.705825 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:57.713094 (+ 7269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:33:57.713109 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:33:57.713137 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:33:57.713216 (+ 79us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:33:57.717459 (+ 4243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:33:57.718509 (+ 1050us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:33:57.718518 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:33:57.718522 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:33:57.718711 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:33:57.719199 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:33:57.719205 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:33:57.719207 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:33:57.719666 (+ 459us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:33:57.719964 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0811 06:33:57.721243 (+ 1279us) server_negotiation.cc:299] Negotiation successful
0811 06:33:57.721501 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 06:33:57.722653 22674 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:33:57.723019 23151 log.cc:1196] T 205353c8754145da9bbc49b9dd2ac784 P 8c88b13171e84962a6de95aeeda55c2c: Deleting WAL directory at /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-1/wal/wals/205353c8754145da9bbc49b9dd2ac784
I20260811 06:33:57.723774 23151 ts_tablet_manager.cc:1939] T 205353c8754145da9bbc49b9dd2ac784 P 8c88b13171e84962a6de95aeeda55c2c: Deleting consensus metadata
I20260811 06:33:57.726336 22380 catalog_manager.cc:4841] TS 8c88b13171e84962a6de95aeeda55c2c (127.28.239.194:42877): tablet 205353c8754145da9bbc49b9dd2ac784 (table test-table [id=fa13dd6dd8ac432c963022ab3d38fd96]) successfully deleted
I20260811 06:33:57.727415 23154 tablet_replica.cc:330] T 205353c8754145da9bbc49b9dd2ac784 P 1648c41809414c8f97307d271b2541d6: stopping tablet replica
I20260811 06:33:57.727954 23154 raft_consensus.cc:2229] T 205353c8754145da9bbc49b9dd2ac784 P 1648c41809414c8f97307d271b2541d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:33:57.728543 23154 raft_consensus.cc:2258] T 205353c8754145da9bbc49b9dd2ac784 P 1648c41809414c8f97307d271b2541d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:33:57.731309 23154 ts_tablet_manager.cc:1905] T 205353c8754145da9bbc49b9dd2ac784 P 1648c41809414c8f97307d271b2541d6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:33:57.736505 23154 ts_tablet_manager.cc:1918] T 205353c8754145da9bbc49b9dd2ac784 P 1648c41809414c8f97307d271b2541d6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:33:57.736663 23154 log.cc:1196] T 205353c8754145da9bbc49b9dd2ac784 P 1648c41809414c8f97307d271b2541d6: Deleting WAL directory at /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-2/wal/wals/205353c8754145da9bbc49b9dd2ac784
I20260811 06:33:57.737154 23154 ts_tablet_manager.cc:1939] T 205353c8754145da9bbc49b9dd2ac784 P 1648c41809414c8f97307d271b2541d6: Deleting consensus metadata
I20260811 06:33:57.738881 22375 catalog_manager.cc:4841] TS 1648c41809414c8f97307d271b2541d6 (127.28.239.195:45519): tablet 205353c8754145da9bbc49b9dd2ac784 (table test-table [id=fa13dd6dd8ac432c963022ab3d38fd96]) successfully deleted
I20260811 06:33:57.825574 23152 raft_consensus.cc:2258] T 205353c8754145da9bbc49b9dd2ac784 P 5e51526ceea347ba88b10bc1a50d84ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:33:57.826098 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 22450
W20260811 06:33:57.851295 22380 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.28.239.193:42485 (error 108)
I20260811 06:33:57.851588 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 22687
I20260811 06:33:57.853400 23113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:57.852980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.193:42485 (local address 127.0.0.1:47028)
0811 06:33:57.853151 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:57.853256 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.193:42485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260811 06:33:57.853703 22380 catalog_manager.cc:4584] TS 5e51526ceea347ba88b10bc1a50d84ca (127.28.239.193:42485): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 205353c8754145da9bbc49b9dd2ac784: Network error: Client connection negotiation failed: client connection to 127.28.239.193:42485: connect: Connection refused (error 111)
I20260811 06:33:57.872072 23116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:57.852349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52598 (local address 127.28.239.194:42877)
0811 06:33:57.852605 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0811 06:33:57.852612 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:33:57.852668 (+ 56us) server_negotiation.cc:407] Connection header received
0811 06:33:57.852923 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:33:57.852928 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:33:57.852998 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:33:57.853171 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:33:57.854868 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:57.855674 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:57.857150 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:57.857640 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:57.857789 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:57.865025 (+ 7236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:33:57.865041 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:33:57.865057 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:33:57.865118 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:33:57.868519 (+ 3401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:33:57.869346 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:33:57.869361 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:33:57.869364 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:33:57.869502 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:33:57.870223 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:33:57.870228 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:33:57.870234 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:33:57.870731 (+ 497us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:33:57.871053 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0811 06:33:57.871420 (+ 367us) server_negotiation.cc:299] Negotiation successful
0811 06:33:57.871671 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260811 06:33:57.872869 22865 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:33:57.907441 23113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:57.906881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.193:42485 (local address 127.0.0.1:47044)
0811 06:33:57.907109 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:57.907257 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.193:42485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260811 06:33:57.908620 23113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:57.908199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.193:42485 (local address 127.0.0.1:47052)
0811 06:33:57.908368 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:57.908476 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.193:42485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260811 06:33:57.964380 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 22687
I20260811 06:33:57.969472 23113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:57.968841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.193:42485 (local address 127.0.0.1:47054)
0811 06:33:57.969124 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:57.969257 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.193:42485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260811 06:33:57.970820 23113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:57.970381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.193:42485 (local address 127.0.0.1:47066)
0811 06:33:57.970571 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:57.970665 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.193:42485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260811 06:33:57.995518 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 22877
I20260811 06:33:58.010038 23109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:57.996335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51448 (local address 127.28.239.195:45519)
0811 06:33:57.996661 (+ 326us) server_negotiation.cc:206] Beginning negotiation
0811 06:33:57.996669 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:33:57.996757 (+ 88us) server_negotiation.cc:407] Connection header received
0811 06:33:57.996975 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:33:57.996985 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:33:57.997052 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:33:57.997177 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:33:57.998608 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:57.999365 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:58.000677 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:33:58.001087 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:33:58.001235 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:33:58.004044 (+ 2809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:33:58.004060 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:33:58.004078 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:33:58.004113 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:33:58.006996 (+ 2883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:33:58.007788 (+ 792us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:33:58.007796 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:33:58.007799 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:33:58.007942 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:33:58.008347 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:33:58.008353 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:33:58.008356 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:33:58.008736 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:33:58.009052 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0811 06:33:58.009371 (+ 319us) server_negotiation.cc:299] Negotiation successful
0811 06:33:58.009668 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179}
I20260811 06:33:58.010773 23076 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:33:58.051735 23113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:58.051232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.193:42485 (local address 127.0.0.1:47070)
0811 06:33:58.051474 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:58.051570 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.193:42485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260811 06:33:58.052850 23113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:58.052432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.193:42485 (local address 127.0.0.1:47080)
0811 06:33:58.052654 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:58.052727 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.193:42485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260811 06:33:58.103013 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 22877
I20260811 06:33:58.133726 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 22354
I20260811 06:33:58.134979 22439 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:33:58.252041 23113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:33:58.251292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.193:42485 (local address 127.0.0.1:47096)
0811 06:33:58.251676 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0811 06:33:58.251807 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.193:42485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193}
I20260811 06:33:58.252206 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 22354
I20260811 06:34:18.283067 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:42189
--webserver_interface=127.28.239.254
--webserver_port=37769
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab
--principal=kudu/127.28.239.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:43401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.239.254:42189
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:34:18.659626 23161 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:34:18.659982 23161 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:34:18.660147 23161 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:34:18.667449 23161 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 06:34:18.667578 23161 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 06:34:18.667691 23161 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 06:34:18.667780 23161 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:34:18.667878 23161 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:34:18.668004 23161 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:34:18.668099 23161 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:34:18.668203 23161 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:34:18.678508 23161 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:43401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.28.239.254:42189
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab
--principal=kudu/127.28.239.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:42189
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.239.254
--webserver_port=37769
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:34:18.679946 23161 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:34:18.681435 23161 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:34:18.694952 23169 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:34:18.695039 23172 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:34:18.695086 23170 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:34:18.695815 23161 server_base.cc:1034] running on GCE node
I20260811 06:34:18.696617 23161 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:34:18.698187 23161 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:34:18.699414 23161 hybrid_clock.cc:648] HybridClock initialized: now 1786430058699317 us; error 69 us; skew 500 ppm
I20260811 06:34:18.699723 23161 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:34:18 dist-test-slave-gkw7 krb5kdc[21544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430058, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:34:18.707965 23161 init.cc:377] Logged in from keytab as kudu/127.28.239.254@KRBTEST.COM (short username kudu)
I20260811 06:34:18.709740 23161 webserver.cc:460] Webserver started at http://127.28.239.254:37769/ using document root <none> and password file <none>
I20260811 06:34:18.710527 23161 fs_manager.cc:359] Metadata directory not provided
I20260811 06:34:18.710623 23161 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:34:18.716377 23161 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.006s
I20260811 06:34:18.719317 23183 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:34:18.720831 23161 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 06:34:18.721124 23161 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/master-0/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/master-0/wal
uuid: "f9315cc436a5414ea115e916586d7497"
format_stamp: "Formatted at 2026-08-11 06:33:32 on dist-test-slave-gkw7"
server_key: "7f5f786b8c78515694591e9397b4c3e3"
server_key_iv: "198e4780229f0fb00f84ac5a2090c859"
server_key_version: "encryptionkey@0"
I20260811 06:34:18.722265 23161 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:34:18.743971 23161 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:34:18.748287 23161 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:34:18.748621 23161 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:34:18.768697 23161 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.254:42189
I20260811 06:34:18.768718 23263 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.254:42189 every 8 connection(s)
I20260811 06:34:18.770318 23161 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
I20260811 06:34:18.777437 23264 sys_catalog.cc:263] Verifying existing consensus state
I20260811 06:34:18.779104 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 23161
I20260811 06:34:18.780198 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.193:42485
--local_ip_for_outbound_sockets=127.28.239.193
--tserver_master_addrs=127.28.239.254:42189
--webserver_port=42879
--webserver_interface=127.28.239.193
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.193.keytab
--principal=kudu/127.28.239.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:43401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 06:34:18.782276 23264 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9315cc436a5414ea115e916586d7497: Bootstrap starting.
I20260811 06:34:18.807544 23264 log.cc:824] T 00000000000000000000000000000000 P f9315cc436a5414ea115e916586d7497: Log is configured to *not* fsync() on all Append() calls
I20260811 06:34:18.824249 23264 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9315cc436a5414ea115e916586d7497: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 06:34:18.824847 23264 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9315cc436a5414ea115e916586d7497: Bootstrap complete.
I20260811 06:34:18.835037 23264 raft_consensus.cc:348] T 00000000000000000000000000000000 P f9315cc436a5414ea115e916586d7497 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9315cc436a5414ea115e916586d7497" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 42189 } }
I20260811 06:34:18.835964 23264 raft_consensus.cc:729] T 00000000000000000000000000000000 P f9315cc436a5414ea115e916586d7497 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9315cc436a5414ea115e916586d7497, State: Initialized, Role: FOLLOWER
I20260811 06:34:18.836621 23264 consensus_queue.cc:260] T 00000000000000000000000000000000 P f9315cc436a5414ea115e916586d7497 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9315cc436a5414ea115e916586d7497" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 42189 } }
I20260811 06:34:18.836953 23264 raft_consensus.cc:388] T 00000000000000000000000000000000 P f9315cc436a5414ea115e916586d7497 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:34:18.837081 23264 raft_consensus.cc:482] T 00000000000000000000000000000000 P f9315cc436a5414ea115e916586d7497 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:34:18.837294 23264 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f9315cc436a5414ea115e916586d7497 [term 2 FOLLOWER]: Advancing to term 3
I20260811 06:34:18.841224 23264 raft_consensus.cc:504] T 00000000000000000000000000000000 P f9315cc436a5414ea115e916586d7497 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9315cc436a5414ea115e916586d7497" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 42189 } }
I20260811 06:34:18.841805 23264 leader_election.cc:302] T 00000000000000000000000000000000 P f9315cc436a5414ea115e916586d7497 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f9315cc436a5414ea115e916586d7497; no voters:
I20260811 06:34:18.842597 23264 leader_election.cc:288] T 00000000000000000000000000000000 P f9315cc436a5414ea115e916586d7497 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 06:34:18.842753 23268 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f9315cc436a5414ea115e916586d7497 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 06:34:18.844063 23268 raft_consensus.cc:686] T 00000000000000000000000000000000 P f9315cc436a5414ea115e916586d7497 [term 3 LEADER]: Becoming Leader. State: Replica: f9315cc436a5414ea115e916586d7497, State: Running, Role: LEADER
I20260811 06:34:18.844710 23268 consensus_queue.cc:237] T 00000000000000000000000000000000 P f9315cc436a5414ea115e916586d7497 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9315cc436a5414ea115e916586d7497" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 42189 } }
I20260811 06:34:18.845080 23264 sys_catalog.cc:564] T 00000000000000000000000000000000 P f9315cc436a5414ea115e916586d7497 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:34:18.847460 23271 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9315cc436a5414ea115e916586d7497 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f9315cc436a5414ea115e916586d7497. Latest consensus state: current_term: 3 leader_uuid: "f9315cc436a5414ea115e916586d7497" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9315cc436a5414ea115e916586d7497" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 42189 } } }
I20260811 06:34:18.848356 23271 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9315cc436a5414ea115e916586d7497 [sys.catalog]: This master's current role is: LEADER
I20260811 06:34:18.847690 23270 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9315cc436a5414ea115e916586d7497 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "f9315cc436a5414ea115e916586d7497" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9315cc436a5414ea115e916586d7497" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 42189 } } }
I20260811 06:34:18.848738 23270 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9315cc436a5414ea115e916586d7497 [sys.catalog]: This master's current role is: LEADER
I20260811 06:34:18.865299 23276 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:34:18.876324 23276 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:34:18.878551 23276 catalog_manager.cc:1259] Loaded cluster ID: 8ea31dfa1ea24bde8f419dac748e64f0
I20260811 06:34:18.878800 23276 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:34:18.884518 23276 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:34:18.887241 23276 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f9315cc436a5414ea115e916586d7497: Loaded TSK: 0
I20260811 06:34:18.887566 23276 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f9315cc436a5414ea115e916586d7497: Loaded TSK: 1
F20260811 06:34:18.889060 23276 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7ff076e699ad google::LogMessage::Fail() at ??:0
@ 0x7ff076e6c88a google::LogMessage::SendToLog() at ??:0
@ 0x7ff076e694de google::LogMessage::Flush() at ??:0
I20260811 06:34:18.899300 23296 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
@ 0x7ff076e6aa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7ff07abcd7d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7ff07abbc73e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7ff08c89c1c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7ff08c89b783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7ff08c9272a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7ff08c9270c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7ff08c9715e9 std::function<>::operator()() at ??:0
@ 0x7ff08c89fcdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7ff08c89dc36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7ff08c925610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7ff08c925439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7ff079ab5345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7ff079ab891c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7ff079ab8799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7ff079a93eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7ff07cd6d6db start_thread at ??:0
@ 0x7ff075d0361f clone at ??:0
W20260811 06:34:19.230237 23266 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:34:19.230530 23266 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:34:19.230639 23266 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:34:19.238253 23266 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:34:19.238376 23266 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:34:19.238494 23266 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.193
I20260811 06:34:19.248662 23266 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:43401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.193.keytab
--principal=kudu/127.28.239.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.193:42485
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.239.193
--webserver_port=42879
--tserver_master_addrs=127.28.239.254:42189
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.193
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:34:19.250033 23266 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:34:19.251386 23266 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:34:19.265651 23341 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:34:19.265748 23340 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:34:19.265702 23344 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:34:19.266660 23266 server_base.cc:1034] running on GCE node
I20260811 06:34:19.267336 23266 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:34:19.268378 23266 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:34:19.269626 23266 hybrid_clock.cc:648] HybridClock initialized: now 1786430059269580 us; error 48 us; skew 500 ppm
I20260811 06:34:19.269976 23266 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:34:19 dist-test-slave-gkw7 krb5kdc[21544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430059, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:34:19.273984 23266 init.cc:377] Logged in from keytab as kudu/127.28.239.193@KRBTEST.COM (short username kudu)
I20260811 06:34:19.277585 23266 webserver.cc:460] Webserver started at http://127.28.239.193:42879/ using document root <none> and password file <none>
I20260811 06:34:19.278596 23266 fs_manager.cc:359] Metadata directory not provided
I20260811 06:34:19.278771 23266 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:34:19.285877 23266 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 06:34:19.289095 23355 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:34:19.290756 23266 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:34:19.290954 23266 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-0/wal
uuid: "5e51526ceea347ba88b10bc1a50d84ca"
format_stamp: "Formatted at 2026-08-11 06:33:32 on dist-test-slave-gkw7"
server_key: "5cf5d515f041520e8b8894e158130968"
server_key_iv: "9bf9a52c50f3d66cdf0d1c57c1f9e85f"
server_key_version: "encryptionkey@0"
I20260811 06:34:19.292030 23266 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:34:19.306418 23266 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:34:19.310765 23266 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:34:19.311075 23266 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:34:19.312176 23266 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:34:19.314194 23368 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260811 06:34:19.320305 23266 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 06:34:19.320475 23266 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.007s user 0.001s sys 0.000s
I20260811 06:34:19.320658 23266 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 06:34:19.324608 23266 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 06:34:19.324713 23266 ts_tablet_manager.cc:589] Time spent register tablets: real 0.004s user 0.000s sys 0.001s
I20260811 06:34:19.325161 23368 tablet_bootstrap.cc:492] T 205353c8754145da9bbc49b9dd2ac784 P 5e51526ceea347ba88b10bc1a50d84ca: Bootstrap starting.
I20260811 06:34:19.357008 23368 log.cc:824] T 205353c8754145da9bbc49b9dd2ac784 P 5e51526ceea347ba88b10bc1a50d84ca: Log is configured to *not* fsync() on all Append() calls
I20260811 06:34:19.370725 23266 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.193:42485
I20260811 06:34:19.370784 23541 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.193:42485 every 8 connection(s)
I20260811 06:34:19.372064 23368 tablet_bootstrap.cc:492] T 205353c8754145da9bbc49b9dd2ac784 P 5e51526ceea347ba88b10bc1a50d84ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 06:34:19.372069 23266 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
I20260811 06:34:19.372826 23368 tablet_bootstrap.cc:492] T 205353c8754145da9bbc49b9dd2ac784 P 5e51526ceea347ba88b10bc1a50d84ca: Bootstrap complete.
I20260811 06:34:19.374861 23368 ts_tablet_manager.cc:1397] T 205353c8754145da9bbc49b9dd2ac784 P 5e51526ceea347ba88b10bc1a50d84ca: Time spent bootstrapping tablet: real 0.050s user 0.028s sys 0.019s
I20260811 06:34:19.378111 23545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:19.375651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:42189 (local address 127.28.239.193:52547)
0811 06:34:19.376608 (+ 957us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:19.376961 (+ 353us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:42189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":642,"thread_start_us":439,"threads_started":1}
W20260811 06:34:19.378813 23542 heartbeater.cc:644] Failed to heartbeat to 127.28.239.254:42189 (0 consecutive failures): Network error: Failed to ping master at 127.28.239.254:42189: Client connection negotiation failed: client connection to 127.28.239.254:42189: connect: Connection refused (error 111)
I20260811 06:34:19.379196 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 23266
I20260811 06:34:19.379809 23545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:19.379435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:42189 (local address 127.28.239.193:49889)
0811 06:34:19.379593 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:19.379689 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:42189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260811 06:34:19.380081 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.194:42877
--local_ip_for_outbound_sockets=127.28.239.194
--tserver_master_addrs=127.28.239.254:42189
--webserver_port=40013
--webserver_interface=127.28.239.194
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.194.keytab
--principal=kudu/127.28.239.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:43401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 06:34:19.380906 23545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:19.380489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:42189 (local address 127.28.239.193:35505)
0811 06:34:19.380661 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:19.380763 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:42189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260811 06:34:19.381173 23542 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 06:34:19.386166 23368 raft_consensus.cc:348] T 205353c8754145da9bbc49b9dd2ac784 P 5e51526ceea347ba88b10bc1a50d84ca [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e51526ceea347ba88b10bc1a50d84ca" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 42485 } } peers { permanent_uuid: "8c88b13171e84962a6de95aeeda55c2c" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 42877 } } peers { permanent_uuid: "1648c41809414c8f97307d271b2541d6" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 45519 } }
I20260811 06:34:19.387849 23368 raft_consensus.cc:729] T 205353c8754145da9bbc49b9dd2ac784 P 5e51526ceea347ba88b10bc1a50d84ca [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e51526ceea347ba88b10bc1a50d84ca, State: Initialized, Role: FOLLOWER
I20260811 06:34:19.388691 23368 consensus_queue.cc:260] T 205353c8754145da9bbc49b9dd2ac784 P 5e51526ceea347ba88b10bc1a50d84ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e51526ceea347ba88b10bc1a50d84ca" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 42485 } } peers { permanent_uuid: "8c88b13171e84962a6de95aeeda55c2c" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 42877 } } peers { permanent_uuid: "1648c41809414c8f97307d271b2541d6" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 45519 } }
I20260811 06:34:19.390692 23545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:19.390284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:42189 (local address 127.28.239.193:59295)
0811 06:34:19.390457 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:19.390546 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:42189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260811 06:34:19.390888 23368 ts_tablet_manager.cc:1428] T 205353c8754145da9bbc49b9dd2ac784 P 5e51526ceea347ba88b10bc1a50d84ca: Time spent starting tablet: real 0.016s user 0.015s sys 0.001s
I20260811 06:34:19.391893 23545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:19.391544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:42189 (local address 127.28.239.193:41825)
0811 06:34:19.391694 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:19.391771 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:42189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260811 06:34:19.738857 23547 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:34:19.739164 23547 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:34:19.739341 23547 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:34:19.746755 23547 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:34:19.746870 23547 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:34:19.747066 23547 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.194
I20260811 06:34:19.756984 23547 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:43401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.194.keytab
--principal=kudu/127.28.239.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.194:42877
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.239.194
--webserver_port=40013
--tserver_master_addrs=127.28.239.254:42189
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.194
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:34:19.758409 23547 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:34:19.759791 23547 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:34:19.773819 23555 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:34:19.773787 23560 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:34:19.773890 23557 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:34:19.774816 23547 server_base.cc:1034] running on GCE node
I20260811 06:34:19.775499 23547 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:34:19.776531 23547 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:34:19.777765 23547 hybrid_clock.cc:648] HybridClock initialized: now 1786430059777736 us; error 73 us; skew 500 ppm
I20260811 06:34:19.778111 23547 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:34:19 dist-test-slave-gkw7 krb5kdc[21544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430059, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:34:19.781338 23547 init.cc:377] Logged in from keytab as kudu/127.28.239.194@KRBTEST.COM (short username kudu)
I20260811 06:34:19.783025 23547 webserver.cc:460] Webserver started at http://127.28.239.194:40013/ using document root <none> and password file <none>
I20260811 06:34:19.783763 23547 fs_manager.cc:359] Metadata directory not provided
I20260811 06:34:19.783918 23547 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:34:19.789911 23547 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260811 06:34:19.792817 23571 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:34:19.794343 23547 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:34:19.794541 23547 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-1/wal
uuid: "8c88b13171e84962a6de95aeeda55c2c"
format_stamp: "Formatted at 2026-08-11 06:33:33 on dist-test-slave-gkw7"
server_key: "49fbc2a75d3880e5bb58a29c326e0db9"
server_key_iv: "8e68b52c34c37c885ab9340e5c75e9cd"
server_key_version: "encryptionkey@0"
I20260811 06:34:19.795606 23547 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:34:19.824460 23547 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:34:19.829154 23547 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:34:19.829411 23547 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:34:19.830514 23547 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:34:19.831955 23547 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:34:19.832053 23547 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:34:19.832129 23547 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:34:19.832191 23547 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:34:19.867585 23547 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.194:42877
I20260811 06:34:19.867686 23739 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.194:42877 every 8 connection(s)
I20260811 06:34:19.869098 23547 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
I20260811 06:34:19.874272 23742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:19.872248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:42189 (local address 127.28.239.194:46823)
0811 06:34:19.873002 (+ 754us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:19.873293 (+ 291us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:42189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":285,"threads_started":1}
W20260811 06:34:19.874946 23740 heartbeater.cc:644] Failed to heartbeat to 127.28.239.254:42189 (0 consecutive failures): Network error: Failed to ping master at 127.28.239.254:42189: Client connection negotiation failed: client connection to 127.28.239.254:42189: connect: Connection refused (error 111)
I20260811 06:34:19.875900 23742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:19.875530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:42189 (local address 127.28.239.194:47625)
0811 06:34:19.875715 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:19.875786 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:42189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260811 06:34:19.876761 23742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:19.876422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:42189 (local address 127.28.239.194:47063)
0811 06:34:19.876566 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:19.876649 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:42189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260811 06:34:19.877030 23740 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 06:34:19.877470 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 23547
I20260811 06:34:19.878121 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.195:45519
--local_ip_for_outbound_sockets=127.28.239.195
--tserver_master_addrs=127.28.239.254:42189
--webserver_port=41801
--webserver_interface=127.28.239.195
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.195.keytab
--principal=kudu/127.28.239.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:43401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:34:20.230654 23744 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:34:20.230980 23744 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:34:20.231156 23744 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:34:20.238404 23744 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:34:20.238556 23744 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:34:20.238740 23744 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.195
I20260811 06:34:20.248418 23744 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:43401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.195.keytab
--principal=kudu/127.28.239.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.195:45519
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.239.195
--webserver_port=41801
--tserver_master_addrs=127.28.239.254:42189
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.195
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:34:20.249853 23744 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:34:20.251206 23744 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:34:20.265029 23750 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:34:20.265097 23753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:34:20.265074 23759 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:34:20.265944 23744 server_base.cc:1034] running on GCE node
I20260811 06:34:20.266652 23744 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:34:20.267637 23744 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:34:20.268846 23744 hybrid_clock.cc:648] HybridClock initialized: now 1786430060268797 us; error 51 us; skew 500 ppm
I20260811 06:34:20.269177 23744 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:34:20 dist-test-slave-gkw7 krb5kdc[21544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430060, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:34:20.272397 23744 init.cc:377] Logged in from keytab as kudu/127.28.239.195@KRBTEST.COM (short username kudu)
I20260811 06:34:20.274047 23744 webserver.cc:460] Webserver started at http://127.28.239.195:41801/ using document root <none> and password file <none>
I20260811 06:34:20.274786 23744 fs_manager.cc:359] Metadata directory not provided
I20260811 06:34:20.274942 23744 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:34:20.280567 23744 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 06:34:20.283463 23770 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:34:20.284899 23744 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 06:34:20.285084 23744 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-2/wal
uuid: "1648c41809414c8f97307d271b2541d6"
format_stamp: "Formatted at 2026-08-11 06:33:34 on dist-test-slave-gkw7"
server_key: "9091360401284f9d006d7621f18df1ff"
server_key_iv: "0647b0b07517332b949870b3d5b6b892"
server_key_version: "encryptionkey@0"
I20260811 06:34:20.286178 23744 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:34:20.297497 23744 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:34:20.301919 23744 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:34:20.302217 23744 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:34:20.303220 23744 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:34:20.304628 23744 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:34:20.304770 23744 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:34:20.304917 23744 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:34:20.305011 23744 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:34:20.341985 23744 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.195:45519
I20260811 06:34:20.342110 23958 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.195:45519 every 8 connection(s)
I20260811 06:34:20.343577 23744 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
I20260811 06:34:20.348898 23964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:20.347034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:42189 (local address 127.28.239.195:50941)
0811 06:34:20.347791 (+ 757us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:20.348036 (+ 245us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:42189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":286,"threads_started":1}
W20260811 06:34:20.349695 23959 heartbeater.cc:644] Failed to heartbeat to 127.28.239.254:42189 (0 consecutive failures): Network error: Failed to ping master at 127.28.239.254:42189: Client connection negotiation failed: client connection to 127.28.239.254:42189: connect: Connection refused (error 111)
I20260811 06:34:20.350580 23964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:20.350267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:42189 (local address 127.28.239.195:54017)
0811 06:34:20.350409 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:20.350476 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:42189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260811 06:34:20.351526 23964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:20.351225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:42189 (local address 127.28.239.195:37663)
0811 06:34:20.351348 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:20.351418 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:42189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260811 06:34:20.351527 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 23744
W20260811 06:34:20.351841 23959 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 06:34:20.393841 23966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:20.392864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:42189 (local address 127.28.239.193:37321)
0811 06:34:20.393431 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:20.393523 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:42189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":272,"threads_started":1}
I20260811 06:34:20.780315 23967 raft_consensus.cc:482] T 205353c8754145da9bbc49b9dd2ac784 P 5e51526ceea347ba88b10bc1a50d84ca [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:34:20.780696 23967 raft_consensus.cc:504] T 205353c8754145da9bbc49b9dd2ac784 P 5e51526ceea347ba88b10bc1a50d84ca [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e51526ceea347ba88b10bc1a50d84ca" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 42485 } } peers { permanent_uuid: "8c88b13171e84962a6de95aeeda55c2c" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 42877 } } peers { permanent_uuid: "1648c41809414c8f97307d271b2541d6" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 45519 } }
I20260811 06:34:20.782547 23967 leader_election.cc:288] T 205353c8754145da9bbc49b9dd2ac784 P 5e51526ceea347ba88b10bc1a50d84ca [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c88b13171e84962a6de95aeeda55c2c (127.28.239.194:42877), 1648c41809414c8f97307d271b2541d6 (127.28.239.195:45519)
Aug 11 06:34:20 dist-test-slave-gkw7 krb5kdc[21544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430059, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.193@KRBTEST.COM for kudu/127.28.239.194@KRBTEST.COM
Aug 11 06:34:20 dist-test-slave-gkw7 krb5kdc[21544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430059, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.193@KRBTEST.COM for kudu/127.28.239.195@KRBTEST.COM
I20260811 06:34:20.807662 23968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:20.782968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.195:45519 (local address 127.28.239.193:46913)
0811 06:34:20.785111 (+ 2143us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:20.785147 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 06:34:20.785288 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:34:20.788374 (+ 3086us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:34:20.788388 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:34:20.789312 (+ 924us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:34:20.790324 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:20.790346 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:20.792008 (+ 1662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:20.792012 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:20.793463 (+ 1451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:20.793476 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:20.795062 (+ 1586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:20.795071 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:20.795501 (+ 430us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:20.796738 (+ 1237us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:34:20.796764 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:34:20.799729 (+ 2965us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:34:20.803744 (+ 4015us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:34:20.803757 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:34:20.803772 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:34:20.804352 (+ 580us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:34:20.804858 (+ 506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:34:20.804861 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:34:20.804863 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:34:20.805033 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:34:20.805916 (+ 883us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:34:20.805933 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:34:20.806242 (+ 309us) client_negotiation.cc:769] Sending connection context
0811 06:34:20.806531 (+ 289us) client_negotiation.cc:240] Negotiation successful
0811 06:34:20.807145 (+ 614us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1970,"mutex_wait_us":281,"spinlock_wait_cycles":24576,"thread_start_us":1723,"threads_started":1}
I20260811 06:34:20.807703 23966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:20.782802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.194:42877 (local address 127.28.239.193:39413)
0811 06:34:20.783408 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:20.783457 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 06:34:20.784116 (+ 659us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:34:20.788376 (+ 4260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:34:20.788386 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:34:20.789318 (+ 932us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:34:20.790323 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:20.790346 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:20.792013 (+ 1667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:20.792017 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:20.793461 (+ 1444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:20.793475 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:20.795049 (+ 1574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:20.795102 (+ 53us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:20.795496 (+ 394us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:20.796616 (+ 1120us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:34:20.796675 (+ 59us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:34:20.799500 (+ 2825us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:34:20.803805 (+ 4305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:34:20.803809 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:34:20.803811 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:34:20.804341 (+ 530us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:34:20.804813 (+ 472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:34:20.804818 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:34:20.804822 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:34:20.805013 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:34:20.805917 (+ 904us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:34:20.805925 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:34:20.806241 (+ 316us) client_negotiation.cc:769] Sending connection context
0811 06:34:20.806529 (+ 288us) client_negotiation.cc:240] Negotiation successful
0811 06:34:20.807297 (+ 768us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":473,"mutex_wait_us":464,"spinlock_wait_cycles":14976}
I20260811 06:34:20.808614 23969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:20.783416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.193:39413 (local address 127.28.239.194:42877)
0811 06:34:20.785822 (+ 2406us) server_negotiation.cc:206] Beginning negotiation
0811 06:34:20.785839 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 06:34:20.785892 (+ 53us) server_negotiation.cc:407] Connection header received
0811 06:34:20.787444 (+ 1552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:34:20.787455 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:34:20.787868 (+ 413us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:34:20.788233 (+ 365us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:34:20.790543 (+ 2310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:20.791807 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:20.793688 (+ 1881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:20.794792 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:20.794981 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:20.799937 (+ 4956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:34:20.799950 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:34:20.799972 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:34:20.800033 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:34:20.803666 (+ 3633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:34:20.804503 (+ 837us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:34:20.804511 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:34:20.804517 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:34:20.804689 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:34:20.805144 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:34:20.805160 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:34:20.805169 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:34:20.805771 (+ 602us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:34:20.805981 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0811 06:34:20.806785 (+ 804us) server_negotiation.cc:299] Negotiation successful
0811 06:34:20.807203 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2236,"thread_start_us":2127,"threads_started":1}
I20260811 06:34:20.808634 23971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:20.783499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.193:46913 (local address 127.28.239.195:45519)
0811 06:34:20.786061 (+ 2562us) server_negotiation.cc:206] Beginning negotiation
0811 06:34:20.786072 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 06:34:20.786108 (+ 36us) server_negotiation.cc:407] Connection header received
0811 06:34:20.787578 (+ 1470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:34:20.787588 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:34:20.787929 (+ 341us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:34:20.788286 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:34:20.790529 (+ 2243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:20.791821 (+ 1292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:20.793820 (+ 1999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:20.794792 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:20.794988 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:20.800043 (+ 5055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:34:20.800053 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:34:20.800065 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:34:20.800121 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:34:20.803567 (+ 3446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:34:20.804527 (+ 960us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:34:20.804534 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:34:20.804540 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:34:20.804707 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:34:20.805186 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:34:20.805190 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:34:20.805192 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:34:20.805771 (+ 579us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:34:20.805986 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0811 06:34:20.806781 (+ 795us) server_negotiation.cc:299] Negotiation successful
0811 06:34:20.807526 (+ 745us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2389,"thread_start_us":2272,"threads_started":1}
W20260811 06:34:20.812057 23362 leader_election.cc:341] T 205353c8754145da9bbc49b9dd2ac784 P 5e51526ceea347ba88b10bc1a50d84ca [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8c88b13171e84962a6de95aeeda55c2c (127.28.239.194:42877): Not found: Tablet not found: 205353c8754145da9bbc49b9dd2ac784
W20260811 06:34:20.812307 23357 leader_election.cc:341] T 205353c8754145da9bbc49b9dd2ac784 P 5e51526ceea347ba88b10bc1a50d84ca [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1648c41809414c8f97307d271b2541d6 (127.28.239.195:45519): Not found: Tablet not found: 205353c8754145da9bbc49b9dd2ac784
I20260811 06:34:20.812495 23357 leader_election.cc:302] T 205353c8754145da9bbc49b9dd2ac784 P 5e51526ceea347ba88b10bc1a50d84ca [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e51526ceea347ba88b10bc1a50d84ca; no voters: 1648c41809414c8f97307d271b2541d6, 8c88b13171e84962a6de95aeeda55c2c
I20260811 06:34:20.812940 23967 raft_consensus.cc:2726] T 205353c8754145da9bbc49b9dd2ac784 P 5e51526ceea347ba88b10bc1a50d84ca [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 06:34:20.878780 23972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:20.877718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:42189 (local address 127.28.239.194:37127)
0811 06:34:20.878374 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:20.878536 (+ 162us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:42189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":401,"threads_started":1}
I20260811 06:34:21.353192 23973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:21.352388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:42189 (local address 127.28.239.195:36361)
0811 06:34:21.352874 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:21.352986 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:42189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":240,"threads_started":1}
I20260811 06:34:21.395726 23974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:21.394885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:42189 (local address 127.28.239.193:59001)
0811 06:34:21.395385 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:21.395489 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:42189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":244,"threads_started":1}
I20260811 06:34:21.880656 23976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:21.879866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:42189 (local address 127.28.239.194:42017)
0811 06:34:21.880361 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:21.880460 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:42189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":259,"threads_started":1}
I20260811 06:34:22.354892 23977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:22.354162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:42189 (local address 127.28.239.195:35401)
0811 06:34:22.354620 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:22.354713 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:42189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":257,"threads_started":1}
I20260811 06:34:22.397809 23978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:22.396761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:42189 (local address 127.28.239.193:54783)
0811 06:34:22.397407 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:22.397510 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:42189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":531,"thread_start_us":280,"threads_started":1}
I20260811 06:34:22.409785 23979 raft_consensus.cc:482] T 205353c8754145da9bbc49b9dd2ac784 P 5e51526ceea347ba88b10bc1a50d84ca [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:34:22.410024 23979 raft_consensus.cc:504] T 205353c8754145da9bbc49b9dd2ac784 P 5e51526ceea347ba88b10bc1a50d84ca [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e51526ceea347ba88b10bc1a50d84ca" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 42485 } } peers { permanent_uuid: "8c88b13171e84962a6de95aeeda55c2c" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 42877 } } peers { permanent_uuid: "1648c41809414c8f97307d271b2541d6" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 45519 } }
I20260811 06:34:22.410938 23979 leader_election.cc:288] T 205353c8754145da9bbc49b9dd2ac784 P 5e51526ceea347ba88b10bc1a50d84ca [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c88b13171e84962a6de95aeeda55c2c (127.28.239.194:42877), 1648c41809414c8f97307d271b2541d6 (127.28.239.195:45519)
W20260811 06:34:22.412302 23362 leader_election.cc:341] T 205353c8754145da9bbc49b9dd2ac784 P 5e51526ceea347ba88b10bc1a50d84ca [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8c88b13171e84962a6de95aeeda55c2c (127.28.239.194:42877): Not found: Tablet not found: 205353c8754145da9bbc49b9dd2ac784
W20260811 06:34:22.412465 23357 leader_election.cc:341] T 205353c8754145da9bbc49b9dd2ac784 P 5e51526ceea347ba88b10bc1a50d84ca [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1648c41809414c8f97307d271b2541d6 (127.28.239.195:45519): Not found: Tablet not found: 205353c8754145da9bbc49b9dd2ac784
I20260811 06:34:22.412601 23357 leader_election.cc:302] T 205353c8754145da9bbc49b9dd2ac784 P 5e51526ceea347ba88b10bc1a50d84ca [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e51526ceea347ba88b10bc1a50d84ca; no voters: 1648c41809414c8f97307d271b2541d6, 8c88b13171e84962a6de95aeeda55c2c
I20260811 06:34:22.412896 23979 raft_consensus.cc:2726] T 205353c8754145da9bbc49b9dd2ac784 P 5e51526ceea347ba88b10bc1a50d84ca [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 06:34:22.882668 23982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:22.881752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:42189 (local address 127.28.239.194:39645)
0811 06:34:22.882349 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:22.882479 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:42189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":306,"threads_started":1}
I20260811 06:34:23.356715 23985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:23.355936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:42189 (local address 127.28.239.195:44557)
0811 06:34:23.356441 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:23.356535 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:42189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":260,"threads_started":1}
I20260811 06:34:23.399688 23989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:23.398828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:42189 (local address 127.28.239.193:47253)
0811 06:34:23.399415 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:23.399506 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:42189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":342,"threads_started":1}
I20260811 06:34:23.884346 23990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:23.883568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:42189 (local address 127.28.239.194:33827)
0811 06:34:23.884060 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:23.884157 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:42189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":250,"threads_started":1}
I20260811 06:34:24.358538 23991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:24.357740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:42189 (local address 127.28.239.195:58943)
0811 06:34:24.358232 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:24.358318 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:42189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":232,"threads_started":1}
I20260811 06:34:24.401793 23992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:24.400980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:42189 (local address 127.28.239.193:56749)
0811 06:34:24.401462 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:24.401575 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:42189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":260,"threads_started":1}
I20260811 06:34:24.886346 23993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:24.885368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:42189 (local address 127.28.239.194:45351)
0811 06:34:24.886056 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:24.886151 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:42189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":400,"threads_started":1}
I20260811 06:34:25.360247 23997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:25.359527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:42189 (local address 127.28.239.195:43443)
0811 06:34:25.359984 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:25.360072 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:42189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":249,"threads_started":1}
I20260811 06:34:25.403743 23998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:25.402833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:42189 (local address 127.28.239.193:44423)
0811 06:34:25.403440 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:25.403550 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:42189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":340,"threads_started":1}
I20260811 06:34:25.680501 24001 raft_consensus.cc:482] T 205353c8754145da9bbc49b9dd2ac784 P 5e51526ceea347ba88b10bc1a50d84ca [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:34:25.680800 24001 raft_consensus.cc:504] T 205353c8754145da9bbc49b9dd2ac784 P 5e51526ceea347ba88b10bc1a50d84ca [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e51526ceea347ba88b10bc1a50d84ca" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 42485 } } peers { permanent_uuid: "8c88b13171e84962a6de95aeeda55c2c" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 42877 } } peers { permanent_uuid: "1648c41809414c8f97307d271b2541d6" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 45519 } }
I20260811 06:34:25.681844 24001 leader_election.cc:288] T 205353c8754145da9bbc49b9dd2ac784 P 5e51526ceea347ba88b10bc1a50d84ca [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c88b13171e84962a6de95aeeda55c2c (127.28.239.194:42877), 1648c41809414c8f97307d271b2541d6 (127.28.239.195:45519)
W20260811 06:34:25.682941 23362 leader_election.cc:341] T 205353c8754145da9bbc49b9dd2ac784 P 5e51526ceea347ba88b10bc1a50d84ca [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8c88b13171e84962a6de95aeeda55c2c (127.28.239.194:42877): Not found: Tablet not found: 205353c8754145da9bbc49b9dd2ac784
W20260811 06:34:25.683168 23357 leader_election.cc:341] T 205353c8754145da9bbc49b9dd2ac784 P 5e51526ceea347ba88b10bc1a50d84ca [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1648c41809414c8f97307d271b2541d6 (127.28.239.195:45519): Not found: Tablet not found: 205353c8754145da9bbc49b9dd2ac784
I20260811 06:34:25.683319 23357 leader_election.cc:302] T 205353c8754145da9bbc49b9dd2ac784 P 5e51526ceea347ba88b10bc1a50d84ca [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e51526ceea347ba88b10bc1a50d84ca; no voters: 1648c41809414c8f97307d271b2541d6, 8c88b13171e84962a6de95aeeda55c2c
I20260811 06:34:25.683622 24001 raft_consensus.cc:2726] T 205353c8754145da9bbc49b9dd2ac784 P 5e51526ceea347ba88b10bc1a50d84ca [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 06:34:25.888175 24002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:25.887334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:42189 (local address 127.28.239.194:33011)
0811 06:34:25.887858 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:25.887954 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:42189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":283,"threads_started":1}
I20260811 06:34:26.362483 24003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:26.361235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:42189 (local address 127.28.239.195:59115)
0811 06:34:26.362150 (+ 915us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:26.362278 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:42189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":592,"thread_start_us":437,"threads_started":1}
I20260811 06:34:26.405797 24004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:26.404837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:42189 (local address 127.28.239.193:33815)
0811 06:34:26.405373 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:26.405481 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:42189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":289,"threads_started":1}
I20260811 06:34:26.890377 24005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:26.889399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:42189 (local address 127.28.239.194:53697)
0811 06:34:26.890032 (+ 633us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:26.890120 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:42189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":322,"threads_started":1}
I20260811 06:34:27.364490 24006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:27.363547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:42189 (local address 127.28.239.195:34547)
0811 06:34:27.364153 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:27.364268 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:42189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":260,"threads_started":1}
I20260811 06:34:27.407725 24007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:27.406798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:42189 (local address 127.28.239.193:54493)
0811 06:34:27.407458 (+ 660us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:27.407559 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:42189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":557,"thread_start_us":431,"threads_started":1}
I20260811 06:34:27.892258 24011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:27.891372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:42189 (local address 127.28.239.194:50707)
0811 06:34:27.891894 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:27.891991 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:42189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":272,"threads_started":1}
I20260811 06:34:28.366550 24013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:28.365704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:42189 (local address 127.28.239.195:41577)
0811 06:34:28.366224 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:28.366320 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:42189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":262,"threads_started":1}
I20260811 06:34:28.409837 24014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:28.408819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:42189 (local address 127.28.239.193:52077)
0811 06:34:28.409359 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:28.409451 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:42189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":294,"threads_started":1}
I20260811 06:34:28.450174 24016 raft_consensus.cc:482] T 205353c8754145da9bbc49b9dd2ac784 P 5e51526ceea347ba88b10bc1a50d84ca [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:34:28.450477 24016 raft_consensus.cc:504] T 205353c8754145da9bbc49b9dd2ac784 P 5e51526ceea347ba88b10bc1a50d84ca [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e51526ceea347ba88b10bc1a50d84ca" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 42485 } } peers { permanent_uuid: "8c88b13171e84962a6de95aeeda55c2c" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 42877 } } peers { permanent_uuid: "1648c41809414c8f97307d271b2541d6" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 45519 } }
I20260811 06:34:28.451298 24016 leader_election.cc:288] T 205353c8754145da9bbc49b9dd2ac784 P 5e51526ceea347ba88b10bc1a50d84ca [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c88b13171e84962a6de95aeeda55c2c (127.28.239.194:42877), 1648c41809414c8f97307d271b2541d6 (127.28.239.195:45519)
W20260811 06:34:28.452414 23357 leader_election.cc:341] T 205353c8754145da9bbc49b9dd2ac784 P 5e51526ceea347ba88b10bc1a50d84ca [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1648c41809414c8f97307d271b2541d6 (127.28.239.195:45519): Not found: Tablet not found: 205353c8754145da9bbc49b9dd2ac784
W20260811 06:34:28.452694 23362 leader_election.cc:341] T 205353c8754145da9bbc49b9dd2ac784 P 5e51526ceea347ba88b10bc1a50d84ca [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8c88b13171e84962a6de95aeeda55c2c (127.28.239.194:42877): Not found: Tablet not found: 205353c8754145da9bbc49b9dd2ac784
I20260811 06:34:28.452885 23362 leader_election.cc:302] T 205353c8754145da9bbc49b9dd2ac784 P 5e51526ceea347ba88b10bc1a50d84ca [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e51526ceea347ba88b10bc1a50d84ca; no voters: 1648c41809414c8f97307d271b2541d6, 8c88b13171e84962a6de95aeeda55c2c
I20260811 06:34:28.453219 24016 raft_consensus.cc:2726] T 205353c8754145da9bbc49b9dd2ac784 P 5e51526ceea347ba88b10bc1a50d84ca [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 06:34:28.894132 24019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:28.893285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:42189 (local address 127.28.239.194:43367)
0811 06:34:28.893866 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:28.893962 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:42189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":315,"threads_started":1}
I20260811 06:34:29.368536 24022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:29.367695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:42189 (local address 127.28.239.195:52529)
0811 06:34:29.368266 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:29.368369 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:42189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":252,"threads_started":1}
I20260811 06:34:29.412142 24024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:29.411164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:42189 (local address 127.28.239.193:59205)
0811 06:34:29.411845 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:29.411949 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:42189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":369,"threads_started":1}
I20260811 06:34:29.896164 24025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:29.895240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:42189 (local address 127.28.239.194:41355)
0811 06:34:29.895870 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:29.895970 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:42189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":356,"threads_started":1}
I20260811 06:34:30.370784 24026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:30.369882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:42189 (local address 127.28.239.195:50159)
0811 06:34:30.370406 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:30.370543 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:42189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":281,"threads_started":1}
I20260811 06:34:30.414180 24028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:30.413266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:42189 (local address 127.28.239.193:53033)
0811 06:34:30.413880 (+ 614us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:30.413976 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:42189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":341,"threads_started":1}
I20260811 06:34:30.898131 24029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:30.897196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:42189 (local address 127.28.239.194:48217)
0811 06:34:30.897807 (+ 611us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:30.897905 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:42189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":263,"threads_started":1}
I20260811 06:34:31.372829 24031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:31.371947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:42189 (local address 127.28.239.195:48129)
0811 06:34:31.372502 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:31.372614 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:42189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":259,"threads_started":1}
I20260811 06:34:31.415946 24032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:31.415129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:42189 (local address 127.28.239.193:42493)
0811 06:34:31.415660 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:31.415760 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:42189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":270,"threads_started":1}
I20260811 06:34:31.900231 24034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:31.899309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:42189 (local address 127.28.239.194:36993)
0811 06:34:31.899868 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:31.899956 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:42189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":287,"threads_started":1}
I20260811 06:34:32.374794 24035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:32.374005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:42189 (local address 127.28.239.195:58195)
0811 06:34:32.374517 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:32.374613 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:42189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":260,"threads_started":1}
I20260811 06:34:32.417853 24037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:32.416921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:42189 (local address 127.28.239.193:59625)
0811 06:34:32.417488 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:32.417636 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:42189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":309,"threads_started":1}
I20260811 06:34:32.902083 24038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:32.901215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:42189 (local address 127.28.239.194:33871)
0811 06:34:32.901805 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:32.901908 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:42189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":261,"threads_started":1}
I20260811 06:34:33.376886 24039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:33.375978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:42189 (local address 127.28.239.195:37671)
0811 06:34:33.376560 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:33.376660 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:42189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":316,"threads_started":1}
I20260811 06:34:33.419763 24040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:33.418938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:42189 (local address 127.28.239.193:50087)
0811 06:34:33.419473 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:33.419567 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:42189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":289,"threads_started":1}
I20260811 06:34:33.903842 24043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:33.903042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:42189 (local address 127.28.239.194:57231)
0811 06:34:33.903534 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:33.903635 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:42189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":252,"threads_started":1}
I20260811 06:34:34.378748 24044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:34.377991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:42189 (local address 127.28.239.195:41301)
0811 06:34:34.378464 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:34.378550 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:42189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":257,"threads_started":1}
I20260811 06:34:34.421986 24046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:34.421058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:42189 (local address 127.28.239.193:41275)
0811 06:34:34.421676 (+ 618us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:34.421775 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:42189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":257,"threads_started":1}
I20260811 06:34:34.905871 24048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:34.905011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:42189 (local address 127.28.239.194:47647)
0811 06:34:34.905500 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:34.905695 (+ 195us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:42189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":257,"threads_started":1}
I20260811 06:34:35.116448 24049 raft_consensus.cc:482] T 205353c8754145da9bbc49b9dd2ac784 P 5e51526ceea347ba88b10bc1a50d84ca [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:34:35.116683 24049 raft_consensus.cc:504] T 205353c8754145da9bbc49b9dd2ac784 P 5e51526ceea347ba88b10bc1a50d84ca [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e51526ceea347ba88b10bc1a50d84ca" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 42485 } } peers { permanent_uuid: "8c88b13171e84962a6de95aeeda55c2c" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 42877 } } peers { permanent_uuid: "1648c41809414c8f97307d271b2541d6" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 45519 } }
I20260811 06:34:35.117365 24049 leader_election.cc:288] T 205353c8754145da9bbc49b9dd2ac784 P 5e51526ceea347ba88b10bc1a50d84ca [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c88b13171e84962a6de95aeeda55c2c (127.28.239.194:42877), 1648c41809414c8f97307d271b2541d6 (127.28.239.195:45519)
W20260811 06:34:35.118428 23362 leader_election.cc:341] T 205353c8754145da9bbc49b9dd2ac784 P 5e51526ceea347ba88b10bc1a50d84ca [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8c88b13171e84962a6de95aeeda55c2c (127.28.239.194:42877): Not found: Tablet not found: 205353c8754145da9bbc49b9dd2ac784
W20260811 06:34:35.118656 23357 leader_election.cc:341] T 205353c8754145da9bbc49b9dd2ac784 P 5e51526ceea347ba88b10bc1a50d84ca [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1648c41809414c8f97307d271b2541d6 (127.28.239.195:45519): Not found: Tablet not found: 205353c8754145da9bbc49b9dd2ac784
I20260811 06:34:35.118793 23357 leader_election.cc:302] T 205353c8754145da9bbc49b9dd2ac784 P 5e51526ceea347ba88b10bc1a50d84ca [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e51526ceea347ba88b10bc1a50d84ca; no voters: 1648c41809414c8f97307d271b2541d6, 8c88b13171e84962a6de95aeeda55c2c
I20260811 06:34:35.119082 24049 raft_consensus.cc:2726] T 205353c8754145da9bbc49b9dd2ac784 P 5e51526ceea347ba88b10bc1a50d84ca [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 06:34:35.355746 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 23266
I20260811 06:34:35.372609 24050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:35.357172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36058 (local address 127.28.239.193:42485)
0811 06:34:35.357816 (+ 644us) server_negotiation.cc:206] Beginning negotiation
0811 06:34:35.357824 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:34:35.357854 (+ 30us) server_negotiation.cc:407] Connection header received
0811 06:34:35.357925 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:34:35.357932 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:34:35.358156 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:34:35.358351 (+ 195us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:34:35.359866 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:35.360893 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:35.362343 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:35.362956 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:35.363114 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:35.366188 (+ 3074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:34:35.366200 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:34:35.366218 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:34:35.366258 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:34:35.369423 (+ 3165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:34:35.370222 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:34:35.370236 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:34:35.370239 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:34:35.370378 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:34:35.370829 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:34:35.370833 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:34:35.370835 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:34:35.371225 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:34:35.371553 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0811 06:34:35.371920 (+ 367us) server_negotiation.cc:299] Negotiation successful
0811 06:34:35.372254 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":479,"thread_start_us":262,"threads_started":1}
I20260811 06:34:35.374221 23534 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:34:35.380789 24052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:35.379995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:42189 (local address 127.28.239.195:57269)
0811 06:34:35.380502 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:35.380625 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.254:42189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":260,"threads_started":1}
I20260811 06:34:35.470269 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 23266
I20260811 06:34:35.500972 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 23547
I20260811 06:34:35.515393 24054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:35.501905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56188 (local address 127.28.239.194:42877)
0811 06:34:35.502674 (+ 769us) server_negotiation.cc:206] Beginning negotiation
0811 06:34:35.502678 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:34:35.502694 (+ 16us) server_negotiation.cc:407] Connection header received
0811 06:34:35.502756 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:34:35.502761 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:34:35.502819 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:34:35.502911 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:34:35.504369 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:35.505116 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:35.506625 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:35.507062 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:35.507162 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:35.510209 (+ 3047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:34:35.510214 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:34:35.510217 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:34:35.510254 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:34:35.512860 (+ 2606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:34:35.513616 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:34:35.513621 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:34:35.513623 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:34:35.513781 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:34:35.514142 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:34:35.514146 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:34:35.514148 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:34:35.514416 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:34:35.514629 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0811 06:34:35.514911 (+ 282us) server_negotiation.cc:299] Negotiation successful
0811 06:34:35.515085 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":664,"thread_start_us":557,"threads_started":1}
I20260811 06:34:35.516182 23732 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:34:35.602627 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 23547
I20260811 06:34:35.631453 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 23744
I20260811 06:34:35.646672 24057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:35.632417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53170 (local address 127.28.239.195:45519)
0811 06:34:35.633008 (+ 591us) server_negotiation.cc:206] Beginning negotiation
0811 06:34:35.633013 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:34:35.633029 (+ 16us) server_negotiation.cc:407] Connection header received
0811 06:34:35.633085 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:34:35.633089 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:34:35.633159 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:34:35.633317 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:34:35.634954 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:35.635930 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:35.637252 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:35.637925 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:35.638037 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:35.640812 (+ 2775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:34:35.640819 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:34:35.640823 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:34:35.640886 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:34:35.643991 (+ 3105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:34:35.644646 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:34:35.644651 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:34:35.644653 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:34:35.644811 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:34:35.645329 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:34:35.645333 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:34:35.645335 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:34:35.645688 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:34:35.645924 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0811 06:34:35.646161 (+ 237us) server_negotiation.cc:299] Negotiation successful
0811 06:34:35.646365 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":464,"thread_start_us":282,"threads_started":1}
I20260811 06:34:35.647392 23953 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:34:35.732914 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 23744
2026-08-11T06:34:35Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (65620 ms)
[----------] 30 tests from SecurityITest (319741 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:34:35 dist-test-slave-gkw7 krb5kdc[24063](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:34:35 dist-test-slave-gkw7 krb5kdc[24063](info): set up 2 sockets
Aug 11 06:34:35 dist-test-slave-gkw7 krb5kdc[24063](info): commencing operation
krb5kdc: starting...
W20260811 06:34:37.091826 29631 mini_kdc.cc:121] Time spent starting KDC: real 1.307s user 0.006s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/krb5kdc/test-user.keytab.
Aug 11 06:34:37 dist-test-slave-gkw7 krb5kdc[24063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430077, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:34:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:34:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.239.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.239.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.239.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab.
Entry for principal kudu/127.28.239.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab.
I20260811 06:34:37.429587 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:40195
--webserver_interface=127.28.239.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab
--principal=kudu/127.28.239.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:38043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.239.254:40195
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:34:37.791235 24083 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:34:37.791540 24083 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:34:37.791718 24083 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:34:37.798967 24083 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:34:37.799083 24083 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:34:37.799190 24083 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:34:37.799304 24083 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:34:37.799383 24083 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:34:37.809401 24083 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:38043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.239.254:40195
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab
--principal=kudu/127.28.239.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:40195
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.239.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:34:37.810832 24083 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:34:37.812237 24083 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:34:37.825804 24091 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:34:37.825757 24095 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:34:37.825709 24093 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:34:37.826651 24083 server_base.cc:1034] running on GCE node
I20260811 06:34:37.827478 24083 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:34:37.828948 24083 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:34:37.830156 24083 hybrid_clock.cc:648] HybridClock initialized: now 1786430077830085 us; error 63 us; skew 500 ppm
I20260811 06:34:37.830497 24083 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:34:37 dist-test-slave-gkw7 krb5kdc[24063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430077, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:34:37.838647 24083 init.cc:377] Logged in from keytab as kudu/127.28.239.254@KRBTEST.COM (short username kudu)
I20260811 06:34:37.840436 24083 webserver.cc:460] Webserver started at http://127.28.239.254:41003/ using document root <none> and password file <none>
I20260811 06:34:37.841254 24083 fs_manager.cc:359] Metadata directory not provided
I20260811 06:34:37.841378 24083 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:34:37.841734 24083 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:34:37.844980 24083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/minicluster-data/master-0/data/instance:
uuid: "a165c244b93741f0bcdac5c3de7c7b87"
format_stamp: "Formatted at 2026-08-11 06:34:37 on dist-test-slave-gkw7"
server_key: "bdcce56e2785ed0fd36422a0bf31d3a9"
server_key_iv: "f380a11e407b54eb17d84b0ee8a25720"
server_key_version: "encryptionkey@0"
I20260811 06:34:37.845835 24083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/minicluster-data/master-0/wal/instance:
uuid: "a165c244b93741f0bcdac5c3de7c7b87"
format_stamp: "Formatted at 2026-08-11 06:34:37 on dist-test-slave-gkw7"
server_key: "bdcce56e2785ed0fd36422a0bf31d3a9"
server_key_iv: "f380a11e407b54eb17d84b0ee8a25720"
server_key_version: "encryptionkey@0"
I20260811 06:34:37.851557 24083 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 06:34:37.855278 24106 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:34:37.856695 24083 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 06:34:37.856894 24083 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/minicluster-data/master-0/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/minicluster-data/master-0/wal
uuid: "a165c244b93741f0bcdac5c3de7c7b87"
format_stamp: "Formatted at 2026-08-11 06:34:37 on dist-test-slave-gkw7"
server_key: "bdcce56e2785ed0fd36422a0bf31d3a9"
server_key_iv: "f380a11e407b54eb17d84b0ee8a25720"
server_key_version: "encryptionkey@0"
I20260811 06:34:37.857086 24083 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:34:37.874012 24083 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:34:37.878613 24083 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:34:37.878945 24083 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:34:37.899784 24083 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.254:40195
I20260811 06:34:37.899818 24180 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.254:40195 every 8 connection(s)
I20260811 06:34:37.901364 24083 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
I20260811 06:34:37.903954 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 24083
I20260811 06:34:37.904187 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/minicluster-data/master-0/wal/instance
I20260811 06:34:37.905027 29631 external_mini_cluster.cc:1428] Setting key 97e6cf440dafc725f94e088a951bf983
I20260811 06:34:37.905822 24182 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:34:37.921386 24182 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a165c244b93741f0bcdac5c3de7c7b87: Bootstrap starting.
Aug 11 06:34:37 dist-test-slave-gkw7 krb5kdc[24063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430077, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:34:37.924814 24182 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a165c244b93741f0bcdac5c3de7c7b87: Neither blocks nor log segments found. Creating new log.
I20260811 06:34:37.926303 24182 log.cc:824] T 00000000000000000000000000000000 P a165c244b93741f0bcdac5c3de7c7b87: Log is configured to *not* fsync() on all Append() calls
I20260811 06:34:37.930078 24182 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a165c244b93741f0bcdac5c3de7c7b87: No bootstrap required, opened a new log
I20260811 06:34:37.932013 24192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:37.907458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48918 (local address 127.28.239.254:40195)
0811 06:34:37.908452 (+ 994us) server_negotiation.cc:206] Beginning negotiation
0811 06:34:37.908467 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 06:34:37.908526 (+ 59us) server_negotiation.cc:407] Connection header received
0811 06:34:37.910782 (+ 2256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:34:37.910807 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:34:37.911218 (+ 411us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:34:37.911583 (+ 365us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:34:37.913616 (+ 2033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:37.914988 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:37.916934 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:37.917766 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:37.917906 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:37.923115 (+ 5209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:34:37.923124 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:34:37.923139 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:34:37.923187 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:34:37.927149 (+ 3962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:34:37.928056 (+ 907us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:34:37.928065 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:34:37.928073 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:34:37.928288 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:34:37.928667 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:34:37.928672 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:34:37.928675 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:34:37.929416 (+ 741us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:34:37.929777 (+ 361us) server_negotiation.cc:1035] Waiting for connection context
0811 06:34:37.930165 (+ 388us) server_negotiation.cc:299] Negotiation successful
0811 06:34:37.930655 (+ 490us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":753,"thread_start_us":456,"threads_started":1}
I20260811 06:34:37.939149 24182 raft_consensus.cc:348] T 00000000000000000000000000000000 P a165c244b93741f0bcdac5c3de7c7b87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a165c244b93741f0bcdac5c3de7c7b87" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 40195 } }
I20260811 06:34:37.939512 24182 raft_consensus.cc:374] T 00000000000000000000000000000000 P a165c244b93741f0bcdac5c3de7c7b87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:34:37.939589 24182 raft_consensus.cc:729] T 00000000000000000000000000000000 P a165c244b93741f0bcdac5c3de7c7b87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a165c244b93741f0bcdac5c3de7c7b87, State: Initialized, Role: FOLLOWER
I20260811 06:34:37.940160 24182 consensus_queue.cc:260] T 00000000000000000000000000000000 P a165c244b93741f0bcdac5c3de7c7b87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a165c244b93741f0bcdac5c3de7c7b87" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 40195 } }
I20260811 06:34:37.940366 24182 raft_consensus.cc:388] T 00000000000000000000000000000000 P a165c244b93741f0bcdac5c3de7c7b87 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:34:37.940471 24182 raft_consensus.cc:482] T 00000000000000000000000000000000 P a165c244b93741f0bcdac5c3de7c7b87 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:34:37.940598 24182 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a165c244b93741f0bcdac5c3de7c7b87 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:34:37.943187 24182 raft_consensus.cc:504] T 00000000000000000000000000000000 P a165c244b93741f0bcdac5c3de7c7b87 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a165c244b93741f0bcdac5c3de7c7b87" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 40195 } }
I20260811 06:34:37.943691 24182 leader_election.cc:302] T 00000000000000000000000000000000 P a165c244b93741f0bcdac5c3de7c7b87 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a165c244b93741f0bcdac5c3de7c7b87; no voters:
I20260811 06:34:37.944442 24182 leader_election.cc:288] T 00000000000000000000000000000000 P a165c244b93741f0bcdac5c3de7c7b87 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:34:37.944559 24196 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a165c244b93741f0bcdac5c3de7c7b87 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:34:37.945365 24196 raft_consensus.cc:686] T 00000000000000000000000000000000 P a165c244b93741f0bcdac5c3de7c7b87 [term 1 LEADER]: Becoming Leader. State: Replica: a165c244b93741f0bcdac5c3de7c7b87, State: Running, Role: LEADER
I20260811 06:34:37.946002 24196 consensus_queue.cc:237] T 00000000000000000000000000000000 P a165c244b93741f0bcdac5c3de7c7b87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a165c244b93741f0bcdac5c3de7c7b87" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 40195 } }
I20260811 06:34:37.946798 24182 sys_catalog.cc:564] T 00000000000000000000000000000000 P a165c244b93741f0bcdac5c3de7c7b87 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:34:37.948988 24197 sys_catalog.cc:455] T 00000000000000000000000000000000 P a165c244b93741f0bcdac5c3de7c7b87 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a165c244b93741f0bcdac5c3de7c7b87" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a165c244b93741f0bcdac5c3de7c7b87" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 40195 } } }
I20260811 06:34:37.949152 24202 sys_catalog.cc:455] T 00000000000000000000000000000000 P a165c244b93741f0bcdac5c3de7c7b87 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a165c244b93741f0bcdac5c3de7c7b87. Latest consensus state: current_term: 1 leader_uuid: "a165c244b93741f0bcdac5c3de7c7b87" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a165c244b93741f0bcdac5c3de7c7b87" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 40195 } } }
I20260811 06:34:37.949509 24202 sys_catalog.cc:458] T 00000000000000000000000000000000 P a165c244b93741f0bcdac5c3de7c7b87 [sys.catalog]: This master's current role is: LEADER
I20260811 06:34:37.949509 24197 sys_catalog.cc:458] T 00000000000000000000000000000000 P a165c244b93741f0bcdac5c3de7c7b87 [sys.catalog]: This master's current role is: LEADER
I20260811 06:34:37.951859 24209 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:34:37.957607 24209 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:34:37.967422 24209 catalog_manager.cc:1347] Generated new cluster ID: a2337a7bafc047cdbed40fb652e7c2a1
I20260811 06:34:37.967633 24209 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:34:38.009449 24209 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:34:38.010914 24209 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:34:38.026669 24209 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a165c244b93741f0bcdac5c3de7c7b87: Generated new TSK 0
I20260811 06:34:38.027575 24209 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.239.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.239.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.239.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.193.keytab.
Entry for principal kudu/127.28.239.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.193.keytab.
I20260811 06:34:38.109344 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.193:0
--local_ip_for_outbound_sockets=127.28.239.193
--webserver_interface=127.28.239.193
--webserver_port=0
--tserver_master_addrs=127.28.239.254:40195
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.193.keytab
--principal=kudu/127.28.239.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:38043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:34:38.473412 24225 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:34:38.473811 24225 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:34:38.473922 24225 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:34:38.481360 24225 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:34:38.481487 24225 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:34:38.481689 24225 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.193
I20260811 06:34:38.491958 24225 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:38043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.193.keytab
--principal=kudu/127.28.239.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.239.193
--webserver_port=0
--tserver_master_addrs=127.28.239.254:40195
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.193
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:34:38.493353 24225 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:34:38.494760 24225 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:34:38.508303 24231 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:34:38.508437 24232 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:34:38.508792 24237 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:34:38.510046 24225 server_base.cc:1034] running on GCE node
I20260811 06:34:38.510699 24225 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:34:38.511731 24225 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:34:38.512938 24225 hybrid_clock.cc:648] HybridClock initialized: now 1786430078512901 us; error 34 us; skew 500 ppm
I20260811 06:34:38.513283 24225 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:34:38 dist-test-slave-gkw7 krb5kdc[24063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430078, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:34:38.516659 24225 init.cc:377] Logged in from keytab as kudu/127.28.239.193@KRBTEST.COM (short username kudu)
I20260811 06:34:38.518308 24225 webserver.cc:460] Webserver started at http://127.28.239.193:39065/ using document root <none> and password file <none>
I20260811 06:34:38.519027 24225 fs_manager.cc:359] Metadata directory not provided
I20260811 06:34:38.519187 24225 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:34:38.519527 24225 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:34:38.522677 24225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/minicluster-data/ts-0/data/instance:
uuid: "27e5fd9a51774eee870cd7c806f4dfc3"
format_stamp: "Formatted at 2026-08-11 06:34:38 on dist-test-slave-gkw7"
server_key: "e8ad921dc66d616927c758aab0fdad5b"
server_key_iv: "0b7335f3e694792153d5f984ce7b790e"
server_key_version: "encryptionkey@0"
I20260811 06:34:38.523495 24225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/minicluster-data/ts-0/wal/instance:
uuid: "27e5fd9a51774eee870cd7c806f4dfc3"
format_stamp: "Formatted at 2026-08-11 06:34:38 on dist-test-slave-gkw7"
server_key: "e8ad921dc66d616927c758aab0fdad5b"
server_key_iv: "0b7335f3e694792153d5f984ce7b790e"
server_key_version: "encryptionkey@0"
I20260811 06:34:38.528939 24225 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 06:34:38.532620 24255 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:34:38.534087 24225 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 06:34:38.534301 24225 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/minicluster-data/ts-0/wal
uuid: "27e5fd9a51774eee870cd7c806f4dfc3"
format_stamp: "Formatted at 2026-08-11 06:34:38 on dist-test-slave-gkw7"
server_key: "e8ad921dc66d616927c758aab0fdad5b"
server_key_iv: "0b7335f3e694792153d5f984ce7b790e"
server_key_version: "encryptionkey@0"
I20260811 06:34:38.534512 24225 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:34:38.553836 24225 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:34:38.558643 24225 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:34:38.558971 24225 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:34:38.560094 24225 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:34:38.561844 24225 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:34:38.561954 24225 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:34:38.562108 24225 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:34:38.562186 24225 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:34:38.598409 24225 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.193:36625
I20260811 06:34:38.598471 24435 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.193:36625 every 8 connection(s)
I20260811 06:34:38.599810 24225 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
I20260811 06:34:38.604303 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 24225
I20260811 06:34:38.604535 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/minicluster-data/ts-0/wal/instance
I20260811 06:34:38.605213 29631 external_mini_cluster.cc:1428] Setting key c287b837ec474b430ded72809ad78771
Aug 11 06:34:38 dist-test-slave-gkw7 krb5kdc[24063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430078, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.193@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:34:38.621970 24438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:38.602648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.193:37303 (local address 127.28.239.254:40195)
0811 06:34:38.603166 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0811 06:34:38.603173 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:34:38.604191 (+ 1018us) server_negotiation.cc:407] Connection header received
0811 06:34:38.606476 (+ 2285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:34:38.606482 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:34:38.606583 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:34:38.606719 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:34:38.608540 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:38.609332 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:38.611002 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:38.611451 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:38.611602 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:38.615584 (+ 3982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:34:38.615590 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:34:38.615593 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:34:38.615631 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:34:38.618430 (+ 2799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:34:38.619257 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:34:38.619269 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:34:38.619278 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:34:38.619408 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:34:38.619903 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:34:38.619907 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:34:38.619909 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:34:38.620224 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:34:38.620421 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0811 06:34:38.621431 (+ 1010us) server_negotiation.cc:299] Negotiation successful
0811 06:34:38.621681 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":245,"threads_started":1}
I20260811 06:34:38.623559 24439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:38.603280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:40195 (local address 127.28.239.193:37303)
0811 06:34:38.604012 (+ 732us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:38.604052 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 06:34:38.606167 (+ 2115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:34:38.606997 (+ 830us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:34:38.607006 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:34:38.607739 (+ 733us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:34:38.608353 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:38.608372 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:38.609497 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:38.609509 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:38.610845 (+ 1336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:38.610858 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:38.611709 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:38.611718 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:38.611975 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:38.612706 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:34:38.612734 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:34:38.615313 (+ 2579us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:34:38.618607 (+ 3294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:34:38.618617 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:34:38.618636 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:34:38.619110 (+ 474us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:34:38.619533 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:34:38.619538 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:34:38.619542 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:34:38.619757 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:34:38.620414 (+ 657us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:34:38.620424 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:34:38.620942 (+ 518us) client_negotiation.cc:769] Sending connection context
0811 06:34:38.621326 (+ 384us) client_negotiation.cc:240] Negotiation successful
0811 06:34:38.622011 (+ 685us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":283,"threads_started":1}
I20260811 06:34:38.625308 24436 heartbeater.cc:347] Connected to a master server at 127.28.239.254:40195
I20260811 06:34:38.625684 24436 heartbeater.cc:464] Registering TS with master...
I20260811 06:34:38.626677 24436 heartbeater.cc:511] Master 127.28.239.254:40195 requested a full tablet report, sending...
I20260811 06:34:38.629045 24130 ts_manager.cc:194] Registered new tserver with Master: 27e5fd9a51774eee870cd7c806f4dfc3 (127.28.239.193:36625)
I20260811 06:34:38.630633 24130 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.239.193@KRBTEST.COM'} at 127.28.239.193:37303
WARNING: no policy specified for kudu/127.28.239.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.239.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.239.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.194.keytab.
Entry for principal kudu/127.28.239.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.194.keytab.
I20260811 06:34:38.680734 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.194:0
--local_ip_for_outbound_sockets=127.28.239.194
--webserver_interface=127.28.239.194
--webserver_port=0
--tserver_master_addrs=127.28.239.254:40195
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.194.keytab
--principal=kudu/127.28.239.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:38043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:34:39.042078 24447 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:34:39.042373 24447 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:34:39.042471 24447 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:34:39.050077 24447 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:34:39.050207 24447 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:34:39.050338 24447 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.194
I20260811 06:34:39.060649 24447 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:38043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.194.keytab
--principal=kudu/127.28.239.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.239.194
--webserver_port=0
--tserver_master_addrs=127.28.239.254:40195
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.194
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:34:39.062189 24447 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:34:39.063525 24447 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:34:39.078171 24457 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:34:39.078246 24455 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:34:39.078269 24460 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:34:39.079010 24447 server_base.cc:1034] running on GCE node
I20260811 06:34:39.079800 24447 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:34:39.080861 24447 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:34:39.082069 24447 hybrid_clock.cc:648] HybridClock initialized: now 1786430079082021 us; error 42 us; skew 500 ppm
I20260811 06:34:39.082448 24447 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:34:39 dist-test-slave-gkw7 krb5kdc[24063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430079, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:34:39.086030 24447 init.cc:377] Logged in from keytab as kudu/127.28.239.194@KRBTEST.COM (short username kudu)
I20260811 06:34:39.087571 24447 webserver.cc:460] Webserver started at http://127.28.239.194:33861/ using document root <none> and password file <none>
I20260811 06:34:39.088332 24447 fs_manager.cc:359] Metadata directory not provided
I20260811 06:34:39.088475 24447 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:34:39.088830 24447 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:34:39.091984 24447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/minicluster-data/ts-1/data/instance:
uuid: "2afb1fdb900a43d3b67c40dff191f901"
format_stamp: "Formatted at 2026-08-11 06:34:39 on dist-test-slave-gkw7"
server_key: "e8f5f35ef37cd2314a09f086f7d7ffe8"
server_key_iv: "b75953fb27a3e8b4d76c074207774f4e"
server_key_version: "encryptionkey@0"
I20260811 06:34:39.092790 24447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/minicluster-data/ts-1/wal/instance:
uuid: "2afb1fdb900a43d3b67c40dff191f901"
format_stamp: "Formatted at 2026-08-11 06:34:39 on dist-test-slave-gkw7"
server_key: "e8f5f35ef37cd2314a09f086f7d7ffe8"
server_key_iv: "b75953fb27a3e8b4d76c074207774f4e"
server_key_version: "encryptionkey@0"
I20260811 06:34:39.098369 24447 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260811 06:34:39.102078 24474 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:34:39.103412 24447 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260811 06:34:39.103600 24447 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/minicluster-data/ts-1/wal
uuid: "2afb1fdb900a43d3b67c40dff191f901"
format_stamp: "Formatted at 2026-08-11 06:34:39 on dist-test-slave-gkw7"
server_key: "e8f5f35ef37cd2314a09f086f7d7ffe8"
server_key_iv: "b75953fb27a3e8b4d76c074207774f4e"
server_key_version: "encryptionkey@0"
I20260811 06:34:39.103811 24447 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:34:39.156160 24447 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:34:39.160750 24447 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:34:39.161123 24447 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:34:39.162302 24447 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:34:39.163738 24447 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:34:39.163847 24447 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:34:39.164014 24447 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:34:39.164091 24447 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:34:39.200366 24447 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.194:37429
I20260811 06:34:39.200424 24647 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.194:37429 every 8 connection(s)
I20260811 06:34:39.201865 24447 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
I20260811 06:34:39.206593 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 24447
I20260811 06:34:39.206795 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/minicluster-data/ts-1/wal/instance
I20260811 06:34:39.207442 29631 external_mini_cluster.cc:1428] Setting key c2dfd974d956f81b6023daacddfdd5c2
Aug 11 06:34:39 dist-test-slave-gkw7 krb5kdc[24063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430079, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.194@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:34:39.226701 24652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:39.204922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.194:49867 (local address 127.28.239.254:40195)
0811 06:34:39.205436 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0811 06:34:39.205444 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:34:39.206634 (+ 1190us) server_negotiation.cc:407] Connection header received
0811 06:34:39.209081 (+ 2447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:34:39.209087 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:34:39.209165 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:34:39.209410 (+ 245us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:34:39.211117 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:39.212354 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:39.214181 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:39.214924 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:39.215071 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:39.219117 (+ 4046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:34:39.219126 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:34:39.219134 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:34:39.219180 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:34:39.223115 (+ 3935us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:34:39.224124 (+ 1009us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:34:39.224128 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:34:39.224131 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:34:39.224261 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:34:39.224850 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:34:39.224853 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:34:39.224855 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:34:39.225122 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:34:39.225231 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 06:34:39.226233 (+ 1002us) server_negotiation.cc:299] Negotiation successful
0811 06:34:39.226415 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":239,"threads_started":1}
I20260811 06:34:39.228714 24653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:39.205603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:40195 (local address 127.28.239.194:49867)
0811 06:34:39.206433 (+ 830us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:39.206488 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 06:34:39.208765 (+ 2277us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:34:39.209672 (+ 907us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:34:39.209684 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:34:39.210306 (+ 622us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:34:39.210920 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:39.210939 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:39.212543 (+ 1604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:39.212548 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:39.213995 (+ 1447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:39.214008 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:39.215181 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:39.215188 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:39.215421 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:39.216189 (+ 768us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:34:39.216220 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:34:39.218852 (+ 2632us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:34:39.223262 (+ 4410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:34:39.223270 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:34:39.223284 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:34:39.223980 (+ 696us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:34:39.224401 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:34:39.224408 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:34:39.224414 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:34:39.224726 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:34:39.225260 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:34:39.225271 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:34:39.225813 (+ 542us) client_negotiation.cc:769] Sending connection context
0811 06:34:39.226147 (+ 334us) client_negotiation.cc:240] Negotiation successful
0811 06:34:39.226806 (+ 659us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":558,"thread_start_us":292,"threads_started":1}
I20260811 06:34:39.230458 24648 heartbeater.cc:347] Connected to a master server at 127.28.239.254:40195
I20260811 06:34:39.230804 24648 heartbeater.cc:464] Registering TS with master...
I20260811 06:34:39.231599 24648 heartbeater.cc:511] Master 127.28.239.254:40195 requested a full tablet report, sending...
I20260811 06:34:39.233196 24130 ts_manager.cc:194] Registered new tserver with Master: 2afb1fdb900a43d3b67c40dff191f901 (127.28.239.194:37429)
I20260811 06:34:39.234095 24130 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.239.194@KRBTEST.COM'} at 127.28.239.194:49867
WARNING: no policy specified for kudu/127.28.239.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.239.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.239.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.195.keytab.
Entry for principal kudu/127.28.239.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.195.keytab.
I20260811 06:34:39.284595 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.195:0
--local_ip_for_outbound_sockets=127.28.239.195
--webserver_interface=127.28.239.195
--webserver_port=0
--tserver_master_addrs=127.28.239.254:40195
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.195.keytab
--principal=kudu/127.28.239.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:38043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 06:34:39.634744 24436 heartbeater.cc:503] Master 127.28.239.254:40195 was elected leader, sending a full tablet report...
W20260811 06:34:39.648396 24657 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:34:39.648674 24657 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:34:39.648770 24657 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:34:39.656138 24657 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:34:39.656255 24657 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:34:39.656370 24657 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.195
I20260811 06:34:39.666289 24657 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:38043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.195.keytab
--principal=kudu/127.28.239.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.239.195
--webserver_port=0
--tserver_master_addrs=127.28.239.254:40195
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.195
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:34:39.667658 24657 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:34:39.669027 24657 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:34:39.683229 24667 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:34:39.683374 24665 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:34:39.683283 24671 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:34:39.684484 24657 server_base.cc:1034] running on GCE node
I20260811 06:34:39.685288 24657 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:34:39.686417 24657 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:34:39.687672 24657 hybrid_clock.cc:648] HybridClock initialized: now 1786430079687602 us; error 63 us; skew 500 ppm
I20260811 06:34:39.688019 24657 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:34:39 dist-test-slave-gkw7 krb5kdc[24063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430079, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:34:39.691362 24657 init.cc:377] Logged in from keytab as kudu/127.28.239.195@KRBTEST.COM (short username kudu)
I20260811 06:34:39.692955 24657 webserver.cc:460] Webserver started at http://127.28.239.195:39003/ using document root <none> and password file <none>
I20260811 06:34:39.693791 24657 fs_manager.cc:359] Metadata directory not provided
I20260811 06:34:39.693950 24657 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:34:39.694304 24657 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:34:39.697420 24657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/minicluster-data/ts-2/data/instance:
uuid: "ad72ab406bac42caa3b3f93df37af440"
format_stamp: "Formatted at 2026-08-11 06:34:39 on dist-test-slave-gkw7"
server_key: "b2f632a5419fe2603faedb70cd83e2e4"
server_key_iv: "750095434d92c53232bf0174d43b8bca"
server_key_version: "encryptionkey@0"
I20260811 06:34:39.698364 24657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/minicluster-data/ts-2/wal/instance:
uuid: "ad72ab406bac42caa3b3f93df37af440"
format_stamp: "Formatted at 2026-08-11 06:34:39 on dist-test-slave-gkw7"
server_key: "b2f632a5419fe2603faedb70cd83e2e4"
server_key_iv: "750095434d92c53232bf0174d43b8bca"
server_key_version: "encryptionkey@0"
I20260811 06:34:39.704053 24657 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260811 06:34:39.708006 24682 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:34:39.709606 24657 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:34:39.709795 24657 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/minicluster-data/ts-2/wal
uuid: "ad72ab406bac42caa3b3f93df37af440"
format_stamp: "Formatted at 2026-08-11 06:34:39 on dist-test-slave-gkw7"
server_key: "b2f632a5419fe2603faedb70cd83e2e4"
server_key_iv: "750095434d92c53232bf0174d43b8bca"
server_key_version: "encryptionkey@0"
I20260811 06:34:39.710022 24657 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:34:39.728565 24657 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:34:39.732834 24657 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:34:39.733172 24657 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:34:39.734292 24657 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:34:39.735707 24657 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:34:39.735819 24657 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:34:39.735980 24657 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:34:39.736057 24657 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:34:39.772163 24657 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.195:44819
I20260811 06:34:39.772213 24849 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.195:44819 every 8 connection(s)
I20260811 06:34:39.773567 24657 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
I20260811 06:34:39.778988 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 24657
I20260811 06:34:39.779206 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/minicluster-data/ts-2/wal/instance
I20260811 06:34:39.779814 29631 external_mini_cluster.cc:1428] Setting key 98dc188f6bb5c84a1584f15ae7a9c8ce
Aug 11 06:34:39 dist-test-slave-gkw7 krb5kdc[24063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430079, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.195@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:34:39.796666 24857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:39.776486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.195:44981 (local address 127.28.239.254:40195)
0811 06:34:39.777056 (+ 570us) server_negotiation.cc:206] Beginning negotiation
0811 06:34:39.777063 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:34:39.778259 (+ 1196us) server_negotiation.cc:407] Connection header received
0811 06:34:39.780822 (+ 2563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:34:39.780830 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:34:39.780923 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:34:39.781059 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:34:39.783248 (+ 2189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:39.784102 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:39.785849 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:39.786350 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:39.786462 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:39.790066 (+ 3604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:34:39.790071 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:34:39.790075 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:34:39.790111 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:34:39.792732 (+ 2621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:34:39.793488 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:34:39.793492 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:34:39.793495 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:34:39.793717 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:34:39.794313 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:34:39.794317 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:34:39.794319 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:34:39.794749 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:34:39.794959 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0811 06:34:39.796065 (+ 1106us) server_negotiation.cc:299] Negotiation successful
0811 06:34:39.796349 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":426,"thread_start_us":267,"threads_started":1}
I20260811 06:34:39.797974 24859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:39.777280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:40195 (local address 127.28.239.195:44981)
0811 06:34:39.778079 (+ 799us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:39.778118 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 06:34:39.780537 (+ 2419us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:34:39.781362 (+ 825us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:34:39.781371 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:34:39.782161 (+ 790us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:34:39.783003 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:39.783030 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:39.784255 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:39.784260 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:39.785650 (+ 1390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:39.785663 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:39.786562 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:39.786567 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:39.786782 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:39.787511 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:34:39.787539 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:34:39.789834 (+ 2295us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:34:39.792865 (+ 3031us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:34:39.792873 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:34:39.792887 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:34:39.793332 (+ 445us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:34:39.793858 (+ 526us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:34:39.793864 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:34:39.793868 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:34:39.794164 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:34:39.794982 (+ 818us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:34:39.794991 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:34:39.795532 (+ 541us) client_negotiation.cc:769] Sending connection context
0811 06:34:39.795894 (+ 362us) client_negotiation.cc:240] Negotiation successful
0811 06:34:39.796575 (+ 681us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":313,"threads_started":1}
I20260811 06:34:39.799587 24855 heartbeater.cc:347] Connected to a master server at 127.28.239.254:40195
I20260811 06:34:39.799952 24855 heartbeater.cc:464] Registering TS with master...
I20260811 06:34:39.800777 24855 heartbeater.cc:511] Master 127.28.239.254:40195 requested a full tablet report, sending...
I20260811 06:34:39.802510 24130 ts_manager.cc:194] Registered new tserver with Master: ad72ab406bac42caa3b3f93df37af440 (127.28.239.195:44819)
I20260811 06:34:39.803423 24130 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.239.195@KRBTEST.COM'} at 127.28.239.195:44981
I20260811 06:34:39.809582 29631 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 06:34:39.829856 24857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:39.815469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48924 (local address 127.28.239.254:40195)
0811 06:34:39.815742 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0811 06:34:39.815752 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:34:39.816191 (+ 439us) server_negotiation.cc:407] Connection header received
0811 06:34:39.816422 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:34:39.816435 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:34:39.816523 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:34:39.816655 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:34:39.818448 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:39.819299 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:39.820827 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:39.821295 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:39.821446 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:39.824161 (+ 2715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:34:39.824166 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:34:39.824170 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:34:39.824205 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:34:39.826986 (+ 2781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:34:39.827705 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:34:39.827712 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:34:39.827716 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:34:39.827851 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:34:39.828261 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:34:39.828269 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:34:39.828272 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:34:39.828628 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:34:39.828887 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0811 06:34:39.829237 (+ 350us) server_negotiation.cc:299] Negotiation successful
0811 06:34:39.829496 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 06:34:39.834383 24130 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48924:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:34:39.842615 24130 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:34:39.871699 24872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:39.855624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.193:36625 (local address 127.0.0.1:39764)
0811 06:34:39.858549 (+ 2925us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:39.858613 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0811 06:34:39.858858 (+ 245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:34:39.860158 (+ 1300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:34:39.860171 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:34:39.860216 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:34:39.860727 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:39.860744 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:39.863634 (+ 2890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:39.863643 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:39.867024 (+ 3381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:39.867049 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:39.870240 (+ 3191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:39.870250 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:39.870553 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:39.870629 (+ 76us) client_negotiation.cc:769] Sending connection context
0811 06:34:39.870814 (+ 185us) client_negotiation.cc:240] Negotiation successful
0811 06:34:39.871104 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2613,"thread_start_us":1578,"threads_started":1}
I20260811 06:34:39.872949 24874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:39.855923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39764 (local address 127.28.239.193:36625)
0811 06:34:39.856683 (+ 760us) server_negotiation.cc:206] Beginning negotiation
0811 06:34:39.856693 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:34:39.859107 (+ 2414us) server_negotiation.cc:407] Connection header received
0811 06:34:39.859246 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:34:39.859257 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:34:39.859555 (+ 298us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:34:39.859745 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:34:39.860983 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:39.863406 (+ 2423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:39.867294 (+ 3888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:39.869444 (+ 2150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:39.869764 (+ 320us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:39.870041 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0811 06:34:39.872264 (+ 2223us) server_negotiation.cc:299] Negotiation successful
0811 06:34:39.872526 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":588,"thread_start_us":418,"threads_started":1}
I20260811 06:34:39.879040 24337 tablet_service.cc:1467] Processing CreateTablet for tablet 56fe52caec354b77a82c3cd15c6ff469 (DEFAULT_TABLE table=test-table [id=0e422cb5d14149469d5c9a3f09b47538]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:34:39.880923 24337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56fe52caec354b77a82c3cd15c6ff469. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:34:39.883389 24881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:39.857633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.194:37429 (local address 127.0.0.1:58720)
0811 06:34:39.874533 (+ 16900us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:39.874548 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 06:34:39.874696 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:34:39.875450 (+ 754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:34:39.875455 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:34:39.875482 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:34:39.875914 (+ 432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:39.875932 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:39.878264 (+ 2332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:39.878269 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:39.880540 (+ 2271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:39.880557 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:39.882527 (+ 1970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:39.882535 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:39.882735 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:39.882774 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 06:34:39.882848 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 06:34:39.882996 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16778,"thread_start_us":16583,"threads_started":1}
I20260811 06:34:39.883818 24872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:39.856053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.195:44819 (local address 127.0.0.1:33650)
0811 06:34:39.873553 (+ 17500us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:39.873580 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 06:34:39.873733 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:34:39.876024 (+ 2291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:34:39.876028 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:34:39.876057 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:34:39.876309 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:39.876323 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:39.879749 (+ 3426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:39.879755 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:39.881427 (+ 1672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:39.881440 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:39.883034 (+ 1594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:39.883039 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:39.883255 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:39.883304 (+ 49us) client_negotiation.cc:769] Sending connection context
0811 06:34:39.883379 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 06:34:39.883525 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17334,"thread_start_us":23446,"threads_started":1}
I20260811 06:34:39.884634 24880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:39.856772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33650 (local address 127.28.239.195:44819)
0811 06:34:39.875355 (+ 18583us) server_negotiation.cc:206] Beginning negotiation
0811 06:34:39.875364 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:34:39.875400 (+ 36us) server_negotiation.cc:407] Connection header received
0811 06:34:39.875498 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:34:39.875507 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:34:39.875766 (+ 259us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:34:39.877312 (+ 1546us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:34:39.877625 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:39.879221 (+ 1596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:39.881621 (+ 2400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:39.882855 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:39.883847 (+ 992us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:39.884064 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0811 06:34:39.884146 (+ 82us) server_negotiation.cc:299] Negotiation successful
0811 06:34:39.884324 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18387,"thread_start_us":18216,"threads_started":1}
I20260811 06:34:39.885828 24882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:39.857685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58720 (local address 127.28.239.194:37429)
0811 06:34:39.872937 (+ 15252us) server_negotiation.cc:206] Beginning negotiation
0811 06:34:39.872950 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 06:34:39.874673 (+ 1723us) server_negotiation.cc:407] Connection header received
0811 06:34:39.874853 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:34:39.874864 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:34:39.875182 (+ 318us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:34:39.876181 (+ 999us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:34:39.876469 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:39.878105 (+ 1636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:39.880724 (+ 2619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:39.882277 (+ 1553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:39.884163 (+ 1886us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:39.884361 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0811 06:34:39.884428 (+ 67us) server_negotiation.cc:299] Negotiation successful
0811 06:34:39.884586 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15004,"thread_start_us":15242,"threads_started":1}
I20260811 06:34:39.889037 24762 tablet_service.cc:1467] Processing CreateTablet for tablet 56fe52caec354b77a82c3cd15c6ff469 (DEFAULT_TABLE table=test-table [id=0e422cb5d14149469d5c9a3f09b47538]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:34:39.889411 24551 tablet_service.cc:1467] Processing CreateTablet for tablet 56fe52caec354b77a82c3cd15c6ff469 (DEFAULT_TABLE table=test-table [id=0e422cb5d14149469d5c9a3f09b47538]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:34:39.890728 24762 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56fe52caec354b77a82c3cd15c6ff469. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:34:39.890729 24551 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56fe52caec354b77a82c3cd15c6ff469. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:34:39.894971 24884 tablet_bootstrap.cc:492] T 56fe52caec354b77a82c3cd15c6ff469 P 27e5fd9a51774eee870cd7c806f4dfc3: Bootstrap starting.
I20260811 06:34:39.898634 24884 tablet_bootstrap.cc:654] T 56fe52caec354b77a82c3cd15c6ff469 P 27e5fd9a51774eee870cd7c806f4dfc3: Neither blocks nor log segments found. Creating new log.
I20260811 06:34:39.899833 24884 log.cc:824] T 56fe52caec354b77a82c3cd15c6ff469 P 27e5fd9a51774eee870cd7c806f4dfc3: Log is configured to *not* fsync() on all Append() calls
I20260811 06:34:39.907541 24884 tablet_bootstrap.cc:492] T 56fe52caec354b77a82c3cd15c6ff469 P 27e5fd9a51774eee870cd7c806f4dfc3: No bootstrap required, opened a new log
I20260811 06:34:39.908229 24884 ts_tablet_manager.cc:1397] T 56fe52caec354b77a82c3cd15c6ff469 P 27e5fd9a51774eee870cd7c806f4dfc3: Time spent bootstrapping tablet: real 0.013s user 0.008s sys 0.000s
I20260811 06:34:39.909619 24890 tablet_bootstrap.cc:492] T 56fe52caec354b77a82c3cd15c6ff469 P 2afb1fdb900a43d3b67c40dff191f901: Bootstrap starting.
I20260811 06:34:39.914749 24888 tablet_bootstrap.cc:492] T 56fe52caec354b77a82c3cd15c6ff469 P ad72ab406bac42caa3b3f93df37af440: Bootstrap starting.
I20260811 06:34:39.916177 24890 tablet_bootstrap.cc:654] T 56fe52caec354b77a82c3cd15c6ff469 P 2afb1fdb900a43d3b67c40dff191f901: Neither blocks nor log segments found. Creating new log.
I20260811 06:34:39.917595 24890 log.cc:824] T 56fe52caec354b77a82c3cd15c6ff469 P 2afb1fdb900a43d3b67c40dff191f901: Log is configured to *not* fsync() on all Append() calls
I20260811 06:34:39.920045 24888 tablet_bootstrap.cc:654] T 56fe52caec354b77a82c3cd15c6ff469 P ad72ab406bac42caa3b3f93df37af440: Neither blocks nor log segments found. Creating new log.
I20260811 06:34:39.921393 24888 log.cc:824] T 56fe52caec354b77a82c3cd15c6ff469 P ad72ab406bac42caa3b3f93df37af440: Log is configured to *not* fsync() on all Append() calls
I20260811 06:34:39.924409 24884 raft_consensus.cc:348] T 56fe52caec354b77a82c3cd15c6ff469 P 27e5fd9a51774eee870cd7c806f4dfc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27e5fd9a51774eee870cd7c806f4dfc3" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 36625 } } peers { permanent_uuid: "ad72ab406bac42caa3b3f93df37af440" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 44819 } } peers { permanent_uuid: "2afb1fdb900a43d3b67c40dff191f901" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37429 } }
I20260811 06:34:39.924960 24884 raft_consensus.cc:374] T 56fe52caec354b77a82c3cd15c6ff469 P 27e5fd9a51774eee870cd7c806f4dfc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:34:39.925107 24884 raft_consensus.cc:729] T 56fe52caec354b77a82c3cd15c6ff469 P 27e5fd9a51774eee870cd7c806f4dfc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27e5fd9a51774eee870cd7c806f4dfc3, State: Initialized, Role: FOLLOWER
I20260811 06:34:39.925889 24884 consensus_queue.cc:260] T 56fe52caec354b77a82c3cd15c6ff469 P 27e5fd9a51774eee870cd7c806f4dfc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27e5fd9a51774eee870cd7c806f4dfc3" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 36625 } } peers { permanent_uuid: "ad72ab406bac42caa3b3f93df37af440" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 44819 } } peers { permanent_uuid: "2afb1fdb900a43d3b67c40dff191f901" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37429 } }
I20260811 06:34:39.927759 24890 tablet_bootstrap.cc:492] T 56fe52caec354b77a82c3cd15c6ff469 P 2afb1fdb900a43d3b67c40dff191f901: No bootstrap required, opened a new log
I20260811 06:34:39.927891 24884 ts_tablet_manager.cc:1428] T 56fe52caec354b77a82c3cd15c6ff469 P 27e5fd9a51774eee870cd7c806f4dfc3: Time spent starting tablet: real 0.019s user 0.015s sys 0.000s
I20260811 06:34:39.928148 24890 ts_tablet_manager.cc:1397] T 56fe52caec354b77a82c3cd15c6ff469 P 2afb1fdb900a43d3b67c40dff191f901: Time spent bootstrapping tablet: real 0.019s user 0.015s sys 0.000s
I20260811 06:34:39.931120 24888 tablet_bootstrap.cc:492] T 56fe52caec354b77a82c3cd15c6ff469 P ad72ab406bac42caa3b3f93df37af440: No bootstrap required, opened a new log
I20260811 06:34:39.931475 24888 ts_tablet_manager.cc:1397] T 56fe52caec354b77a82c3cd15c6ff469 P ad72ab406bac42caa3b3f93df37af440: Time spent bootstrapping tablet: real 0.017s user 0.015s sys 0.000s
I20260811 06:34:39.939380 24890 raft_consensus.cc:348] T 56fe52caec354b77a82c3cd15c6ff469 P 2afb1fdb900a43d3b67c40dff191f901 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27e5fd9a51774eee870cd7c806f4dfc3" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 36625 } } peers { permanent_uuid: "ad72ab406bac42caa3b3f93df37af440" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 44819 } } peers { permanent_uuid: "2afb1fdb900a43d3b67c40dff191f901" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37429 } }
I20260811 06:34:39.939891 24890 raft_consensus.cc:374] T 56fe52caec354b77a82c3cd15c6ff469 P 2afb1fdb900a43d3b67c40dff191f901 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:34:39.939999 24890 raft_consensus.cc:729] T 56fe52caec354b77a82c3cd15c6ff469 P 2afb1fdb900a43d3b67c40dff191f901 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2afb1fdb900a43d3b67c40dff191f901, State: Initialized, Role: FOLLOWER
I20260811 06:34:39.940688 24890 consensus_queue.cc:260] T 56fe52caec354b77a82c3cd15c6ff469 P 2afb1fdb900a43d3b67c40dff191f901 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27e5fd9a51774eee870cd7c806f4dfc3" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 36625 } } peers { permanent_uuid: "ad72ab406bac42caa3b3f93df37af440" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 44819 } } peers { permanent_uuid: "2afb1fdb900a43d3b67c40dff191f901" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37429 } }
I20260811 06:34:39.941926 24648 heartbeater.cc:503] Master 127.28.239.254:40195 was elected leader, sending a full tablet report...
I20260811 06:34:39.942373 24888 raft_consensus.cc:348] T 56fe52caec354b77a82c3cd15c6ff469 P ad72ab406bac42caa3b3f93df37af440 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27e5fd9a51774eee870cd7c806f4dfc3" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 36625 } } peers { permanent_uuid: "ad72ab406bac42caa3b3f93df37af440" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 44819 } } peers { permanent_uuid: "2afb1fdb900a43d3b67c40dff191f901" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37429 } }
I20260811 06:34:39.942720 24890 ts_tablet_manager.cc:1428] T 56fe52caec354b77a82c3cd15c6ff469 P 2afb1fdb900a43d3b67c40dff191f901: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260811 06:34:39.942855 24888 raft_consensus.cc:374] T 56fe52caec354b77a82c3cd15c6ff469 P ad72ab406bac42caa3b3f93df37af440 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:34:39.942963 24888 raft_consensus.cc:729] T 56fe52caec354b77a82c3cd15c6ff469 P ad72ab406bac42caa3b3f93df37af440 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad72ab406bac42caa3b3f93df37af440, State: Initialized, Role: FOLLOWER
I20260811 06:34:39.943578 24888 consensus_queue.cc:260] T 56fe52caec354b77a82c3cd15c6ff469 P ad72ab406bac42caa3b3f93df37af440 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27e5fd9a51774eee870cd7c806f4dfc3" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 36625 } } peers { permanent_uuid: "ad72ab406bac42caa3b3f93df37af440" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 44819 } } peers { permanent_uuid: "2afb1fdb900a43d3b67c40dff191f901" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37429 } }
I20260811 06:34:39.944766 24855 heartbeater.cc:503] Master 127.28.239.254:40195 was elected leader, sending a full tablet report...
I20260811 06:34:39.945402 24888 ts_tablet_manager.cc:1428] T 56fe52caec354b77a82c3cd15c6ff469 P ad72ab406bac42caa3b3f93df37af440: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
W20260811 06:34:39.954221 24651 tablet.cc:2369] T 56fe52caec354b77a82c3cd15c6ff469 P 2afb1fdb900a43d3b67c40dff191f901: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 06:34:40.025398 24856 tablet.cc:2369] T 56fe52caec354b77a82c3cd15c6ff469 P ad72ab406bac42caa3b3f93df37af440: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 06:34:40.102540 24437 tablet.cc:2369] T 56fe52caec354b77a82c3cd15c6ff469 P 27e5fd9a51774eee870cd7c806f4dfc3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:34:40.104427 24897 raft_consensus.cc:482] T 56fe52caec354b77a82c3cd15c6ff469 P ad72ab406bac42caa3b3f93df37af440 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:34:40.104785 24897 raft_consensus.cc:504] T 56fe52caec354b77a82c3cd15c6ff469 P ad72ab406bac42caa3b3f93df37af440 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27e5fd9a51774eee870cd7c806f4dfc3" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 36625 } } peers { permanent_uuid: "ad72ab406bac42caa3b3f93df37af440" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 44819 } } peers { permanent_uuid: "2afb1fdb900a43d3b67c40dff191f901" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37429 } }
I20260811 06:34:40.106755 24897 leader_election.cc:288] T 56fe52caec354b77a82c3cd15c6ff469 P ad72ab406bac42caa3b3f93df37af440 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 27e5fd9a51774eee870cd7c806f4dfc3 (127.28.239.193:36625), 2afb1fdb900a43d3b67c40dff191f901 (127.28.239.194:37429)
I20260811 06:34:40.112915 24859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:40.107052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.193:36625 (local address 127.28.239.195:48935)
0811 06:34:40.107330 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:40.107354 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 06:34:40.107513 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:34:40.108030 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:34:40.108044 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:34:40.108083 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:34:40.108322 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:40.108334 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:40.109511 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:40.109521 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:40.110872 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:40.110888 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:40.112116 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:40.112122 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:40.112343 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:40.112370 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 06:34:40.112483 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 06:34:40.112622 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90}
I20260811 06:34:40.112900 24874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:40.107321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.195:48935 (local address 127.28.239.193:36625)
0811 06:34:40.107579 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0811 06:34:40.107588 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:34:40.107617 (+ 29us) server_negotiation.cc:407] Connection header received
0811 06:34:40.107746 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:34:40.107755 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:34:40.107861 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:34:40.107975 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:34:40.108510 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:40.109356 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:40.111041 (+ 1685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:40.111955 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:40.112128 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:40.112207 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 06:34:40.112511 (+ 304us) server_negotiation.cc:299] Negotiation successful
0811 06:34:40.112642 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 06:34:40.114022 24367 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56fe52caec354b77a82c3cd15c6ff469" candidate_uuid: "ad72ab406bac42caa3b3f93df37af440" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27e5fd9a51774eee870cd7c806f4dfc3" is_pre_election: true
I20260811 06:34:40.114660 24367 raft_consensus.cc:2454] T 56fe52caec354b77a82c3cd15c6ff469 P 27e5fd9a51774eee870cd7c806f4dfc3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad72ab406bac42caa3b3f93df37af440 in term 0.
I20260811 06:34:40.114737 24882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:40.107284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.195:37331 (local address 127.28.239.194:37429)
0811 06:34:40.107539 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0811 06:34:40.107545 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:34:40.108335 (+ 790us) server_negotiation.cc:407] Connection header received
0811 06:34:40.108626 (+ 291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:34:40.108631 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:34:40.108733 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:34:40.108876 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:34:40.109408 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:40.110262 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:40.112292 (+ 2030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:40.113716 (+ 1424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:40.113915 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:40.113994 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 06:34:40.114299 (+ 305us) server_negotiation.cc:299] Negotiation successful
0811 06:34:40.114442 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260811 06:34:40.114926 24899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:40.107052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.194:37429 (local address 127.28.239.195:37331)
0811 06:34:40.108204 (+ 1152us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:40.108226 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 06:34:40.108372 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:34:40.108874 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:34:40.108879 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:34:40.108905 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:34:40.109248 (+ 343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:40.109257 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:40.110428 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:40.110433 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:40.112104 (+ 1671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:40.112118 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:40.113943 (+ 1825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:40.113950 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:40.114123 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:40.114149 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 06:34:40.114268 (+ 119us) client_negotiation.cc:240] Negotiation successful
0811 06:34:40.114409 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1013,"mutex_wait_us":198,"thread_start_us":503,"threads_started":1}
I20260811 06:34:40.115443 24687 leader_election.cc:302] T 56fe52caec354b77a82c3cd15c6ff469 P ad72ab406bac42caa3b3f93df37af440 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27e5fd9a51774eee870cd7c806f4dfc3, ad72ab406bac42caa3b3f93df37af440; no voters:
I20260811 06:34:40.115866 24897 raft_consensus.cc:2781] T 56fe52caec354b77a82c3cd15c6ff469 P ad72ab406bac42caa3b3f93df37af440 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:34:40.116061 24897 raft_consensus.cc:482] T 56fe52caec354b77a82c3cd15c6ff469 P ad72ab406bac42caa3b3f93df37af440 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:34:40.116207 24897 raft_consensus.cc:3037] T 56fe52caec354b77a82c3cd15c6ff469 P ad72ab406bac42caa3b3f93df37af440 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:34:40.116093 24577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56fe52caec354b77a82c3cd15c6ff469" candidate_uuid: "ad72ab406bac42caa3b3f93df37af440" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2afb1fdb900a43d3b67c40dff191f901" is_pre_election: true
I20260811 06:34:40.116741 24577 raft_consensus.cc:2454] T 56fe52caec354b77a82c3cd15c6ff469 P 2afb1fdb900a43d3b67c40dff191f901 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad72ab406bac42caa3b3f93df37af440 in term 0.
I20260811 06:34:40.119468 24897 raft_consensus.cc:504] T 56fe52caec354b77a82c3cd15c6ff469 P ad72ab406bac42caa3b3f93df37af440 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27e5fd9a51774eee870cd7c806f4dfc3" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 36625 } } peers { permanent_uuid: "ad72ab406bac42caa3b3f93df37af440" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 44819 } } peers { permanent_uuid: "2afb1fdb900a43d3b67c40dff191f901" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37429 } }
I20260811 06:34:40.120340 24897 leader_election.cc:288] T 56fe52caec354b77a82c3cd15c6ff469 P ad72ab406bac42caa3b3f93df37af440 [CANDIDATE]: Term 1 election: Requested vote from peers 27e5fd9a51774eee870cd7c806f4dfc3 (127.28.239.193:36625), 2afb1fdb900a43d3b67c40dff191f901 (127.28.239.194:37429)
I20260811 06:34:40.120862 24367 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56fe52caec354b77a82c3cd15c6ff469" candidate_uuid: "ad72ab406bac42caa3b3f93df37af440" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27e5fd9a51774eee870cd7c806f4dfc3"
I20260811 06:34:40.120879 24577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56fe52caec354b77a82c3cd15c6ff469" candidate_uuid: "ad72ab406bac42caa3b3f93df37af440" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2afb1fdb900a43d3b67c40dff191f901"
I20260811 06:34:40.121073 24367 raft_consensus.cc:3037] T 56fe52caec354b77a82c3cd15c6ff469 P 27e5fd9a51774eee870cd7c806f4dfc3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:34:40.121124 24577 raft_consensus.cc:3037] T 56fe52caec354b77a82c3cd15c6ff469 P 2afb1fdb900a43d3b67c40dff191f901 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:34:40.123507 24367 raft_consensus.cc:2454] T 56fe52caec354b77a82c3cd15c6ff469 P 27e5fd9a51774eee870cd7c806f4dfc3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad72ab406bac42caa3b3f93df37af440 in term 1.
I20260811 06:34:40.123696 24577 raft_consensus.cc:2454] T 56fe52caec354b77a82c3cd15c6ff469 P 2afb1fdb900a43d3b67c40dff191f901 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad72ab406bac42caa3b3f93df37af440 in term 1.
I20260811 06:34:40.123966 24687 leader_election.cc:302] T 56fe52caec354b77a82c3cd15c6ff469 P ad72ab406bac42caa3b3f93df37af440 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27e5fd9a51774eee870cd7c806f4dfc3, ad72ab406bac42caa3b3f93df37af440; no voters:
I20260811 06:34:40.124300 24897 raft_consensus.cc:2781] T 56fe52caec354b77a82c3cd15c6ff469 P ad72ab406bac42caa3b3f93df37af440 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:34:40.124764 24897 raft_consensus.cc:686] T 56fe52caec354b77a82c3cd15c6ff469 P ad72ab406bac42caa3b3f93df37af440 [term 1 LEADER]: Becoming Leader. State: Replica: ad72ab406bac42caa3b3f93df37af440, State: Running, Role: LEADER
I20260811 06:34:40.125312 24897 consensus_queue.cc:237] T 56fe52caec354b77a82c3cd15c6ff469 P ad72ab406bac42caa3b3f93df37af440 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27e5fd9a51774eee870cd7c806f4dfc3" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 36625 } } peers { permanent_uuid: "ad72ab406bac42caa3b3f93df37af440" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 44819 } } peers { permanent_uuid: "2afb1fdb900a43d3b67c40dff191f901" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37429 } }
I20260811 06:34:40.130277 24129 catalog_manager.cc:5495] T 56fe52caec354b77a82c3cd15c6ff469 P ad72ab406bac42caa3b3f93df37af440 reported cstate change: term changed from 0 to 1, leader changed from <none> to ad72ab406bac42caa3b3f93df37af440 (127.28.239.195). New cstate: current_term: 1 leader_uuid: "ad72ab406bac42caa3b3f93df37af440" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27e5fd9a51774eee870cd7c806f4dfc3" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 36625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad72ab406bac42caa3b3f93df37af440" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 44819 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2afb1fdb900a43d3b67c40dff191f901" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37429 } health_report { overall_health: UNKNOWN } } }
I20260811 06:34:40.190670 24880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:40.184864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33652 (local address 127.28.239.195:44819)
0811 06:34:40.185052 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 06:34:40.185057 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:34:40.185299 (+ 242us) server_negotiation.cc:407] Connection header received
0811 06:34:40.185472 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:34:40.185476 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:34:40.185595 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:34:40.185725 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:34:40.186088 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:40.186931 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:40.188421 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:40.188839 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:40.188984 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:40.189497 (+ 513us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:34:40.190051 (+ 554us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:34:40.190236 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0811 06:34:40.190300 (+ 64us) server_negotiation.cc:299] Negotiation successful
0811 06:34:40.190382 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 06:34:40.199194 24577 raft_consensus.cc:1261] T 56fe52caec354b77a82c3cd15c6ff469 P 2afb1fdb900a43d3b67c40dff191f901 [term 1 FOLLOWER]: Refusing update from remote peer ad72ab406bac42caa3b3f93df37af440: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:34:40.199194 24367 raft_consensus.cc:1261] T 56fe52caec354b77a82c3cd15c6ff469 P 27e5fd9a51774eee870cd7c806f4dfc3 [term 1 FOLLOWER]: Refusing update from remote peer ad72ab406bac42caa3b3f93df37af440: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:34:40.200327 24900 consensus_queue.cc:1035] T 56fe52caec354b77a82c3cd15c6ff469 P ad72ab406bac42caa3b3f93df37af440 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2afb1fdb900a43d3b67c40dff191f901" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:34:40.200726 24897 consensus_queue.cc:1035] T 56fe52caec354b77a82c3cd15c6ff469 P ad72ab406bac42caa3b3f93df37af440 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27e5fd9a51774eee870cd7c806f4dfc3" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 36625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:34:40.217464 24907 mvcc.cc:205] Tried to move back new op lower bound from 7317217608484548608 to 7317217608197423104. Current Snapshot: MvccSnapshot[applied={T|T < 7317217608484548608}]
I20260811 06:34:40.236348 24130 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48924:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:34:40.236690 24130 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48924:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:34:40.240944 24130 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a165c244b93741f0bcdac5c3de7c7b87: Sending DeleteTablet for 3 replicas of tablet 56fe52caec354b77a82c3cd15c6ff469
I20260811 06:34:40.242378 24762 tablet_service.cc:1514] Processing DeleteTablet for tablet 56fe52caec354b77a82c3cd15c6ff469 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:34:40 UTC) from {username='kudu', principal='kudu/127.28.239.254@KRBTEST.COM'} at 127.0.0.1:33650
I20260811 06:34:40.242372 24337 tablet_service.cc:1514] Processing DeleteTablet for tablet 56fe52caec354b77a82c3cd15c6ff469 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:34:40 UTC) from {username='kudu', principal='kudu/127.28.239.254@KRBTEST.COM'} at 127.0.0.1:39764
I20260811 06:34:40.242504 24551 tablet_service.cc:1514] Processing DeleteTablet for tablet 56fe52caec354b77a82c3cd15c6ff469 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:34:40 UTC) from {username='kudu', principal='kudu/127.28.239.254@KRBTEST.COM'} at 127.0.0.1:58720
I20260811 06:34:40.248768 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 24225
I20260811 06:34:40.253036 24922 tablet_replica.cc:330] T 56fe52caec354b77a82c3cd15c6ff469 P 2afb1fdb900a43d3b67c40dff191f901: stopping tablet replica
I20260811 06:34:40.253590 24922 raft_consensus.cc:2229] T 56fe52caec354b77a82c3cd15c6ff469 P 2afb1fdb900a43d3b67c40dff191f901 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:34:40.254172 24922 raft_consensus.cc:2258] T 56fe52caec354b77a82c3cd15c6ff469 P 2afb1fdb900a43d3b67c40dff191f901 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:34:40.257267 24922 ts_tablet_manager.cc:1905] T 56fe52caec354b77a82c3cd15c6ff469 P 2afb1fdb900a43d3b67c40dff191f901: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:34:40.257982 24920 tablet_replica.cc:330] T 56fe52caec354b77a82c3cd15c6ff469 P ad72ab406bac42caa3b3f93df37af440: stopping tablet replica
I20260811 06:34:40.258502 24920 raft_consensus.cc:2229] T 56fe52caec354b77a82c3cd15c6ff469 P ad72ab406bac42caa3b3f93df37af440 [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:34:40.259282 24920 raft_consensus.cc:2258] T 56fe52caec354b77a82c3cd15c6ff469 P ad72ab406bac42caa3b3f93df37af440 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:34:40.268741 24920 ts_tablet_manager.cc:1905] T 56fe52caec354b77a82c3cd15c6ff469 P ad72ab406bac42caa3b3f93df37af440: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:34:40.270562 24922 ts_tablet_manager.cc:1918] T 56fe52caec354b77a82c3cd15c6ff469 P 2afb1fdb900a43d3b67c40dff191f901: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:34:40.270848 24922 log.cc:1196] T 56fe52caec354b77a82c3cd15c6ff469 P 2afb1fdb900a43d3b67c40dff191f901: Deleting WAL directory at /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/minicluster-data/ts-1/wal/wals/56fe52caec354b77a82c3cd15c6ff469
I20260811 06:34:40.271225 24921 tablet_replica.cc:330] T 56fe52caec354b77a82c3cd15c6ff469 P 27e5fd9a51774eee870cd7c806f4dfc3: stopping tablet replica
Aug 11 06:34:40 dist-test-slave-gkw7 krb5kdc[24063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430077, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.239.193@KRBTEST.COM
I20260811 06:34:40.271538 24922 ts_tablet_manager.cc:1939] T 56fe52caec354b77a82c3cd15c6ff469 P 2afb1fdb900a43d3b67c40dff191f901: Deleting consensus metadata
I20260811 06:34:40.271790 24921 raft_consensus.cc:2229] T 56fe52caec354b77a82c3cd15c6ff469 P 27e5fd9a51774eee870cd7c806f4dfc3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:34:40.274125 24107 catalog_manager.cc:4841] TS 2afb1fdb900a43d3b67c40dff191f901 (127.28.239.194:37429): tablet 56fe52caec354b77a82c3cd15c6ff469 (table test-table [id=0e422cb5d14149469d5c9a3f09b47538]) successfully deleted
I20260811 06:34:40.274834 24921 raft_consensus.cc:2258] T 56fe52caec354b77a82c3cd15c6ff469 P 27e5fd9a51774eee870cd7c806f4dfc3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:34:40.277796 24921 ts_tablet_manager.cc:1905] T 56fe52caec354b77a82c3cd15c6ff469 P 27e5fd9a51774eee870cd7c806f4dfc3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:34:40.280659 24920 ts_tablet_manager.cc:1918] T 56fe52caec354b77a82c3cd15c6ff469 P ad72ab406bac42caa3b3f93df37af440: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:34:40.280900 24920 log.cc:1196] T 56fe52caec354b77a82c3cd15c6ff469 P ad72ab406bac42caa3b3f93df37af440: Deleting WAL directory at /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/minicluster-data/ts-2/wal/wals/56fe52caec354b77a82c3cd15c6ff469
I20260811 06:34:40.281606 24920 ts_tablet_manager.cc:1939] T 56fe52caec354b77a82c3cd15c6ff469 P ad72ab406bac42caa3b3f93df37af440: Deleting consensus metadata
I20260811 06:34:40.281749 24874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:40.250063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39766 (local address 127.28.239.193:36625)
0811 06:34:40.250329 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0811 06:34:40.250338 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:34:40.255016 (+ 4678us) server_negotiation.cc:407] Connection header received
0811 06:34:40.255262 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:34:40.255271 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:34:40.255402 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:34:40.255528 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:34:40.259128 (+ 3600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:40.260723 (+ 1595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:40.263755 (+ 3032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:40.264716 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:40.264848 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:40.273089 (+ 8241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:34:40.273106 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:34:40.273128 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:34:40.273194 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:34:40.277432 (+ 4238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:34:40.278857 (+ 1425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:34:40.278867 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:34:40.278871 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:34:40.279062 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:34:40.279556 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:34:40.279561 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:34:40.279564 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:34:40.280064 (+ 500us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:34:40.280387 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0811 06:34:40.280991 (+ 604us) server_negotiation.cc:299] Negotiation successful
0811 06:34:40.281244 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260811 06:34:40.283411 24430 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:34:40.283896 24110 catalog_manager.cc:4841] TS ad72ab406bac42caa3b3f93df37af440 (127.28.239.195:44819): tablet 56fe52caec354b77a82c3cd15c6ff469 (table test-table [id=0e422cb5d14149469d5c9a3f09b47538]) successfully deleted
I20260811 06:34:40.375636 24921 ts_tablet_manager.cc:1918] T 56fe52caec354b77a82c3cd15c6ff469 P 27e5fd9a51774eee870cd7c806f4dfc3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:34:40.376066 24921 log.cc:1196] T 56fe52caec354b77a82c3cd15c6ff469 P 27e5fd9a51774eee870cd7c806f4dfc3: Deleting WAL directory at /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786429755927508-29631-0/minicluster-data/ts-0/wal/wals/56fe52caec354b77a82c3cd15c6ff469
I20260811 06:34:40.376354 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 24225
W20260811 06:34:40.406549 24109 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.28.239.193:36625 (error 108)
I20260811 06:34:40.407488 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 24447
I20260811 06:34:40.409158 24928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:40.408116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.193:36625 (local address 127.0.0.1:39782)
0811 06:34:40.408734 (+ 618us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:40.408886 (+ 152us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.193:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":489,"threads_started":1}
W20260811 06:34:40.409451 24109 catalog_manager.cc:4584] TS 27e5fd9a51774eee870cd7c806f4dfc3 (127.28.239.193:36625): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 56fe52caec354b77a82c3cd15c6ff469: Network error: Client connection negotiation failed: client connection to 127.28.239.193:36625: connect: Connection refused (error 111)
Aug 11 06:34:40 dist-test-slave-gkw7 krb5kdc[24063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430077, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.239.194@KRBTEST.COM
I20260811 06:34:40.426724 24882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:40.408484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58732 (local address 127.28.239.194:37429)
0811 06:34:40.408689 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 06:34:40.408697 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:34:40.408724 (+ 27us) server_negotiation.cc:407] Connection header received
0811 06:34:40.408806 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:34:40.408814 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:34:40.408931 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:34:40.409056 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:34:40.410926 (+ 1870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:40.411716 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:40.413927 (+ 2211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:40.414361 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:40.414461 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:40.420322 (+ 5861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:34:40.420339 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:34:40.420357 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:34:40.420397 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:34:40.423565 (+ 3168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:34:40.424377 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:34:40.424386 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:34:40.424388 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:34:40.424540 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:34:40.424977 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:34:40.424982 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:34:40.424984 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:34:40.425407 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:34:40.425784 (+ 377us) server_negotiation.cc:1035] Waiting for connection context
0811 06:34:40.426122 (+ 338us) server_negotiation.cc:299] Negotiation successful
0811 06:34:40.426385 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 06:34:40.427486 24642 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:34:40.463225 24928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:40.462653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.193:36625 (local address 127.0.0.1:39784)
0811 06:34:40.462902 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:40.463023 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.193:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260811 06:34:40.464453 24928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:40.463984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.193:36625 (local address 127.0.0.1:39790)
0811 06:34:40.464177 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:40.464269 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.193:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260811 06:34:40.524580 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 24447
I20260811 06:34:40.534209 24928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:40.524740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.193:36625 (local address 127.0.0.1:39800)
0811 06:34:40.533840 (+ 9100us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:40.533983 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.193:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":8897}
I20260811 06:34:40.535729 24928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:40.535197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.193:36625 (local address 127.0.0.1:39806)
0811 06:34:40.535409 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:40.535524 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.193:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260811 06:34:40.548568 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 24657
Aug 11 06:34:40 dist-test-slave-gkw7 krb5kdc[24063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430077, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.239.195@KRBTEST.COM
I20260811 06:34:40.567516 24880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:40.549809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33668 (local address 127.28.239.195:44819)
0811 06:34:40.550032 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0811 06:34:40.550039 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:34:40.550058 (+ 19us) server_negotiation.cc:407] Connection header received
0811 06:34:40.550230 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:34:40.550236 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:34:40.550326 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:34:40.550438 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:34:40.551738 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:40.552588 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:40.554134 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:40.554585 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:40.554762 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:40.559407 (+ 4645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:34:40.559428 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:34:40.559451 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:34:40.559541 (+ 90us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:34:40.564187 (+ 4646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:34:40.564938 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:34:40.564950 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:34:40.564956 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:34:40.565174 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:34:40.565627 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:34:40.565632 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:34:40.565636 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:34:40.566175 (+ 539us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:34:40.566562 (+ 387us) server_negotiation.cc:1035] Waiting for connection context
0811 06:34:40.566776 (+ 214us) server_negotiation.cc:299] Negotiation successful
0811 06:34:40.567039 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 06:34:40.568507 24844 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:34:40.617096 24928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:40.616582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.193:36625 (local address 127.0.0.1:39814)
0811 06:34:40.616837 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:40.616939 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.193:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
I20260811 06:34:40.618511 24928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:40.618054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.193:36625 (local address 127.0.0.1:39824)
0811 06:34:40.618293 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:40.618374 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.193:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136}
I20260811 06:34:40.695205 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 24657
I20260811 06:34:40.724326 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 24083
I20260811 06:34:40.725634 24175 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:34:40.837337 24928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:40.836505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.193:36625 (local address 127.0.0.1:39838)
0811 06:34:40.836955 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:40.837107 (+ 152us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.193:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260811 06:34:40.837589 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 24083
2026-08-11T06:34:40Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (5108 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-11T06:34:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:34:40Z Disabled control of system clock
I20260811 06:34:40.943253 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429755927508-29631-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429755927508-29631-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429755927508-29631-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:44695
--webserver_interface=127.28.239.254
--webserver_port=0
--builtin_ntp_servers=127.28.239.212:39579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.239.254:44695
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 06:34:41.319034 24940 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:34:41.319384 24940 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:34:41.319525 24940 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:34:41.327239 24940 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:34:41.327360 24940 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:34:41.327432 24940 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:34:41.327522 24940 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:34:41.327611 24940 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:34:41.337196 24940 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:39579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429755927508-29631-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429755927508-29631-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.239.254:44695
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:44695
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.239.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429755927508-29631-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:34:41.338631 24940 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:34:41.340549 24940 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:34:41.353580 24946 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:34:41.353504 24947 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:34:41.353582 24950 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:34:41.354763 24940 server_base.cc:1034] running on GCE node
I20260811 06:34:41.355690 24940 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:34:41.357273 24940 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:34:41.358554 24940 hybrid_clock.cc:648] HybridClock initialized: now 1786430081358487 us; error 59 us; skew 500 ppm
I20260811 06:34:41.358982 24940 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:34:41.361639 24940 webserver.cc:460] Webserver started at http://127.28.239.254:37671/ using document root <none> and password file <none>
I20260811 06:34:41.362408 24940 fs_manager.cc:359] Metadata directory not provided
I20260811 06:34:41.362553 24940 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:34:41.362906 24940 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:34:41.366149 24940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429755927508-29631-0/minicluster-data/master-0/data/instance:
uuid: "54e72c9836f947a1a31517960671cfeb"
format_stamp: "Formatted at 2026-08-11 06:34:41 on dist-test-slave-gkw7"
server_key: "b455f4c6fe8c32dc2fe5160fa788d46c"
server_key_iv: "81f94b004a05e20bdd868c7a0e38ce1c"
server_key_version: "encryptionkey@0"
I20260811 06:34:41.367013 24940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429755927508-29631-0/minicluster-data/master-0/wal/instance:
uuid: "54e72c9836f947a1a31517960671cfeb"
format_stamp: "Formatted at 2026-08-11 06:34:41 on dist-test-slave-gkw7"
server_key: "b455f4c6fe8c32dc2fe5160fa788d46c"
server_key_iv: "81f94b004a05e20bdd868c7a0e38ce1c"
server_key_version: "encryptionkey@0"
I20260811 06:34:41.372607 24940 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 06:34:41.376690 24965 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:34:41.378424 24940 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 06:34:41.378650 24940 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429755927508-29631-0/minicluster-data/master-0/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429755927508-29631-0/minicluster-data/master-0/wal
uuid: "54e72c9836f947a1a31517960671cfeb"
format_stamp: "Formatted at 2026-08-11 06:34:41 on dist-test-slave-gkw7"
server_key: "b455f4c6fe8c32dc2fe5160fa788d46c"
server_key_iv: "81f94b004a05e20bdd868c7a0e38ce1c"
server_key_version: "encryptionkey@0"
I20260811 06:34:41.378911 24940 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429755927508-29631-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429755927508-29631-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429755927508-29631-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:34:41.401203 24940 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:34:41.402200 24940 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:34:41.402534 24940 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:34:41.439333 24940 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.254:44695
I20260811 06:34:41.439393 25039 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.254:44695 every 8 connection(s)
I20260811 06:34:41.441435 24940 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
I20260811 06:34:41.447561 25040 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:34:41.449111 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 24940
I20260811 06:34:41.449328 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429755927508-29631-0/minicluster-data/master-0/wal/instance
I20260811 06:34:41.450009 29631 external_mini_cluster.cc:1428] Setting key 9e7fdeecd4a618f605cf3c258da2fe46
I20260811 06:34:41.466073 25040 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54e72c9836f947a1a31517960671cfeb: Bootstrap starting.
I20260811 06:34:41.470901 25040 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 54e72c9836f947a1a31517960671cfeb: Neither blocks nor log segments found. Creating new log.
I20260811 06:34:41.472482 25040 log.cc:824] T 00000000000000000000000000000000 P 54e72c9836f947a1a31517960671cfeb: Log is configured to *not* fsync() on all Append() calls
I20260811 06:34:41.473834 25044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:41.454684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45974 (local address 127.28.239.254:44695)
0811 06:34:41.456320 (+ 1636us) server_negotiation.cc:206] Beginning negotiation
0811 06:34:41.456355 (+ 35us) server_negotiation.cc:399] Waiting for connection header
0811 06:34:41.456425 (+ 70us) server_negotiation.cc:407] Connection header received
0811 06:34:41.459575 (+ 3150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:34:41.459616 (+ 41us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:34:41.460461 (+ 845us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:34:41.461173 (+ 712us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:34:41.461948 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:41.464404 (+ 2456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:41.466780 (+ 2376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:41.467976 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:41.469035 (+ 1059us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:41.469901 (+ 866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:34:41.469912 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:34:41.469932 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:34:41.470723 (+ 791us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:34:41.470888 (+ 165us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:34:41.470921 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:34:41.471743 (+ 822us) server_negotiation.cc:1035] Waiting for connection context
0811 06:34:41.471890 (+ 147us) server_negotiation.cc:299] Negotiation successful
0811 06:34:41.472268 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":965,"thread_start_us":573,"threads_started":1}
I20260811 06:34:41.476406 25040 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54e72c9836f947a1a31517960671cfeb: No bootstrap required, opened a new log
I20260811 06:34:41.487044 25040 raft_consensus.cc:348] T 00000000000000000000000000000000 P 54e72c9836f947a1a31517960671cfeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54e72c9836f947a1a31517960671cfeb" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 44695 } }
I20260811 06:34:41.487576 25040 raft_consensus.cc:374] T 00000000000000000000000000000000 P 54e72c9836f947a1a31517960671cfeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:34:41.487715 25040 raft_consensus.cc:729] T 00000000000000000000000000000000 P 54e72c9836f947a1a31517960671cfeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54e72c9836f947a1a31517960671cfeb, State: Initialized, Role: FOLLOWER
I20260811 06:34:41.488466 25040 consensus_queue.cc:260] T 00000000000000000000000000000000 P 54e72c9836f947a1a31517960671cfeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54e72c9836f947a1a31517960671cfeb" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 44695 } }
I20260811 06:34:41.488818 25040 raft_consensus.cc:388] T 00000000000000000000000000000000 P 54e72c9836f947a1a31517960671cfeb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:34:41.489002 25040 raft_consensus.cc:482] T 00000000000000000000000000000000 P 54e72c9836f947a1a31517960671cfeb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:34:41.489305 25040 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 54e72c9836f947a1a31517960671cfeb [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:34:41.491916 25040 raft_consensus.cc:504] T 00000000000000000000000000000000 P 54e72c9836f947a1a31517960671cfeb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54e72c9836f947a1a31517960671cfeb" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 44695 } }
I20260811 06:34:41.492563 25040 leader_election.cc:302] T 00000000000000000000000000000000 P 54e72c9836f947a1a31517960671cfeb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 54e72c9836f947a1a31517960671cfeb; no voters:
I20260811 06:34:41.493242 25040 leader_election.cc:288] T 00000000000000000000000000000000 P 54e72c9836f947a1a31517960671cfeb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:34:41.493520 25047 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 54e72c9836f947a1a31517960671cfeb [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:34:41.494396 25047 raft_consensus.cc:686] T 00000000000000000000000000000000 P 54e72c9836f947a1a31517960671cfeb [term 1 LEADER]: Becoming Leader. State: Replica: 54e72c9836f947a1a31517960671cfeb, State: Running, Role: LEADER
I20260811 06:34:41.495015 25047 consensus_queue.cc:237] T 00000000000000000000000000000000 P 54e72c9836f947a1a31517960671cfeb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54e72c9836f947a1a31517960671cfeb" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 44695 } }
I20260811 06:34:41.495723 25040 sys_catalog.cc:564] T 00000000000000000000000000000000 P 54e72c9836f947a1a31517960671cfeb [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:34:41.498251 25048 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54e72c9836f947a1a31517960671cfeb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "54e72c9836f947a1a31517960671cfeb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54e72c9836f947a1a31517960671cfeb" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 44695 } } }
I20260811 06:34:41.498402 25049 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54e72c9836f947a1a31517960671cfeb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 54e72c9836f947a1a31517960671cfeb. Latest consensus state: current_term: 1 leader_uuid: "54e72c9836f947a1a31517960671cfeb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54e72c9836f947a1a31517960671cfeb" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 44695 } } }
I20260811 06:34:41.498762 25048 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54e72c9836f947a1a31517960671cfeb [sys.catalog]: This master's current role is: LEADER
I20260811 06:34:41.498848 25049 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54e72c9836f947a1a31517960671cfeb [sys.catalog]: This master's current role is: LEADER
I20260811 06:34:41.499775 25055 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:34:41.507145 25055 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:34:41.516716 25055 catalog_manager.cc:1347] Generated new cluster ID: 55f313c27a684733b173d7894e9f83e6
I20260811 06:34:41.516846 25055 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:34:41.556784 25055 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:34:41.558589 25055 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:34:41.574240 25055 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 54e72c9836f947a1a31517960671cfeb: Generated new TSK 0
I20260811 06:34:41.575392 25055 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:34:41.590444 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429755927508-29631-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429755927508-29631-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429755927508-29631-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.193:0
--local_ip_for_outbound_sockets=127.28.239.193
--webserver_interface=127.28.239.193
--webserver_port=0
--tserver_master_addrs=127.28.239.254:44695
--builtin_ntp_servers=127.28.239.212:39579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 06:34:41.947276 25073 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:34:41.947605 25073 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:34:41.947779 25073 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:34:41.955318 25073 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:34:41.955466 25073 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:34:41.955657 25073 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.193
I20260811 06:34:41.964872 25073 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:39579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429755927508-29631-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429755927508-29631-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.239.193
--webserver_port=0
--tserver_master_addrs=127.28.239.254:44695
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.193
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429755927508-29631-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:34:41.966305 25073 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:34:41.968158 25073 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:34:41.982559 25083 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:34:41.982508 25084 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:34:41.982487 25086 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:34:41.983721 25073 server_base.cc:1034] running on GCE node
I20260811 06:34:41.984525 25073 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:34:41.985759 25073 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:34:41.986986 25073 hybrid_clock.cc:648] HybridClock initialized: now 1786430081986945 us; error 37 us; skew 500 ppm
I20260811 06:34:41.987396 25073 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:34:41.990031 25073 webserver.cc:460] Webserver started at http://127.28.239.193:46721/ using document root <none> and password file <none>
I20260811 06:34:41.990911 25073 fs_manager.cc:359] Metadata directory not provided
I20260811 06:34:41.991046 25073 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:34:41.991466 25073 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:34:41.994858 25073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429755927508-29631-0/minicluster-data/ts-0/data/instance:
uuid: "2ec144d07b6440eca3fca57e36d4d48e"
format_stamp: "Formatted at 2026-08-11 06:34:41 on dist-test-slave-gkw7"
server_key: "2bedc9c15febb150201cdec129b332f2"
server_key_iv: "82bc05c949d8de5dcaaa8d9d0c1e5ebc"
server_key_version: "encryptionkey@0"
I20260811 06:34:41.995837 25073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429755927508-29631-0/minicluster-data/ts-0/wal/instance:
uuid: "2ec144d07b6440eca3fca57e36d4d48e"
format_stamp: "Formatted at 2026-08-11 06:34:41 on dist-test-slave-gkw7"
server_key: "2bedc9c15febb150201cdec129b332f2"
server_key_iv: "82bc05c949d8de5dcaaa8d9d0c1e5ebc"
server_key_version: "encryptionkey@0"
I20260811 06:34:42.002136 25073 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260811 06:34:42.006407 25101 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:34:42.007993 25073 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 06:34:42.008205 25073 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429755927508-29631-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429755927508-29631-0/minicluster-data/ts-0/wal
uuid: "2ec144d07b6440eca3fca57e36d4d48e"
format_stamp: "Formatted at 2026-08-11 06:34:41 on dist-test-slave-gkw7"
server_key: "2bedc9c15febb150201cdec129b332f2"
server_key_iv: "82bc05c949d8de5dcaaa8d9d0c1e5ebc"
server_key_version: "encryptionkey@0"
I20260811 06:34:42.008410 25073 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429755927508-29631-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429755927508-29631-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429755927508-29631-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:34:42.044371 25073 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:34:42.045301 25073 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:34:42.045624 25073 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:34:42.046707 25073 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:34:42.048302 25073 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:34:42.048411 25073 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:34:42.048503 25073 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:34:42.048573 25073 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:34:42.086148 25073 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.193:35535
I20260811 06:34:42.086207 25269 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.193:35535 every 8 connection(s)
I20260811 06:34:42.087584 25073 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
I20260811 06:34:42.095067 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 25073
I20260811 06:34:42.095283 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429755927508-29631-0/minicluster-data/ts-0/wal/instance
I20260811 06:34:42.095875 29631 external_mini_cluster.cc:1428] Setting key 01c7e3eb75c19b7a0a36f4eb039918d8
I20260811 06:34:42.098765 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429755927508-29631-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429755927508-29631-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429755927508-29631-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.194:0
--local_ip_for_outbound_sockets=127.28.239.194
--webserver_interface=127.28.239.194
--webserver_port=0
--tserver_master_addrs=127.28.239.254:44695
--builtin_ntp_servers=127.28.239.212:39579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 06:34:42.101033 25272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:42.090205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.193:45317 (local address 127.28.239.254:44695)
0811 06:34:42.090669 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0811 06:34:42.090674 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:34:42.092115 (+ 1441us) server_negotiation.cc:407] Connection header received
0811 06:34:42.093943 (+ 1828us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:34:42.093947 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:34:42.094019 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:34:42.094121 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:34:42.095215 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:42.096331 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:42.098095 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:42.098528 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:42.098690 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:42.100176 (+ 1486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:34:42.100181 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:34:42.100185 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:34:42.100203 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:34:42.100242 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:34:42.100255 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:34:42.100356 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 06:34:42.100591 (+ 235us) server_negotiation.cc:299] Negotiation successful
0811 06:34:42.100690 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":240,"threads_started":1}
I20260811 06:34:42.102260 25276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:42.090959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:44695 (local address 127.28.239.193:45317)
0811 06:34:42.091950 (+ 991us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:42.091987 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 06:34:42.093697 (+ 1710us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:34:42.094327 (+ 630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:34:42.094335 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:34:42.094422 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:34:42.095025 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:42.095044 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:42.096552 (+ 1508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:42.096556 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:42.097941 (+ 1385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:42.097959 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:42.098774 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:42.098784 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:42.099098 (+ 314us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:42.099849 (+ 751us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 06:34:42.099852 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:34:42.099892 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 06:34:42.099968 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:34:42.100366 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:34:42.100380 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:34:42.100394 (+ 14us) client_negotiation.cc:769] Sending connection context
0811 06:34:42.100581 (+ 187us) client_negotiation.cc:240] Negotiation successful
0811 06:34:42.101008 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":274,"threads_started":1}
I20260811 06:34:42.103964 25270 heartbeater.cc:347] Connected to a master server at 127.28.239.254:44695
I20260811 06:34:42.104285 25270 heartbeater.cc:464] Registering TS with master...
I20260811 06:34:42.105042 25270 heartbeater.cc:511] Master 127.28.239.254:44695 requested a full tablet report, sending...
I20260811 06:34:42.107208 24987 ts_manager.cc:194] Registered new tserver with Master: 2ec144d07b6440eca3fca57e36d4d48e (127.28.239.193:35535)
I20260811 06:34:42.108685 24987 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.28.239.193:45317
W20260811 06:34:42.454908 25278 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:34:42.455190 25278 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:34:42.455279 25278 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:34:42.462934 25278 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:34:42.463044 25278 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:34:42.463157 25278 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.194
I20260811 06:34:42.473023 25278 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:39579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429755927508-29631-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429755927508-29631-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.239.194
--webserver_port=0
--tserver_master_addrs=127.28.239.254:44695
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.194
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429755927508-29631-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:34:42.474398 25278 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:34:42.476150 25278 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:34:42.491155 25288 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:34:42.491307 25291 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:34:42.491559 25287 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:34:42.492430 25278 server_base.cc:1034] running on GCE node
I20260811 06:34:42.493258 25278 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:34:42.494525 25278 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:34:42.495822 25278 hybrid_clock.cc:648] HybridClock initialized: now 1786430082495746 us; error 55 us; skew 500 ppm
I20260811 06:34:42.496311 25278 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:34:42.499168 25278 webserver.cc:460] Webserver started at http://127.28.239.194:44331/ using document root <none> and password file <none>
I20260811 06:34:42.500005 25278 fs_manager.cc:359] Metadata directory not provided
I20260811 06:34:42.500176 25278 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:34:42.500578 25278 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:34:42.504170 25278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429755927508-29631-0/minicluster-data/ts-1/data/instance:
uuid: "6162f379978f42859df5c3793466b4df"
format_stamp: "Formatted at 2026-08-11 06:34:42 on dist-test-slave-gkw7"
server_key: "1de951039b8c78bf444186743eed9da6"
server_key_iv: "98b333a5ccf53a9dce71ebf57ee5bb29"
server_key_version: "encryptionkey@0"
I20260811 06:34:42.505170 25278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429755927508-29631-0/minicluster-data/ts-1/wal/instance:
uuid: "6162f379978f42859df5c3793466b4df"
format_stamp: "Formatted at 2026-08-11 06:34:42 on dist-test-slave-gkw7"
server_key: "1de951039b8c78bf444186743eed9da6"
server_key_iv: "98b333a5ccf53a9dce71ebf57ee5bb29"
server_key_version: "encryptionkey@0"
I20260811 06:34:42.511477 25278 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260811 06:34:42.515537 25299 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:34:42.516964 25278 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 06:34:42.517158 25278 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429755927508-29631-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429755927508-29631-0/minicluster-data/ts-1/wal
uuid: "6162f379978f42859df5c3793466b4df"
format_stamp: "Formatted at 2026-08-11 06:34:42 on dist-test-slave-gkw7"
server_key: "1de951039b8c78bf444186743eed9da6"
server_key_iv: "98b333a5ccf53a9dce71ebf57ee5bb29"
server_key_version: "encryptionkey@0"
I20260811 06:34:42.517387 25278 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429755927508-29631-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429755927508-29631-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429755927508-29631-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:34:42.538638 25278 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:34:42.539578 25278 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:34:42.539928 25278 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:34:42.541069 25278 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:34:42.542843 25278 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:34:42.542934 25278 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:34:42.543098 25278 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:34:42.543186 25278 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:34:42.580363 25278 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.194:40885
I20260811 06:34:42.580500 25469 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.194:40885 every 8 connection(s)
I20260811 06:34:42.581879 25278 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
I20260811 06:34:42.588454 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 25278
I20260811 06:34:42.588665 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429755927508-29631-0/minicluster-data/ts-1/wal/instance
I20260811 06:34:42.589161 29631 external_mini_cluster.cc:1428] Setting key 37c37b29b1a652956e6bac5e14c7b78c
I20260811 06:34:42.592365 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429755927508-29631-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429755927508-29631-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429755927508-29631-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.195:0
--local_ip_for_outbound_sockets=127.28.239.195
--webserver_interface=127.28.239.195
--webserver_port=0
--tserver_master_addrs=127.28.239.254:44695
--builtin_ntp_servers=127.28.239.212:39579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 06:34:42.596505 25272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:42.584972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.194:35913 (local address 127.28.239.254:44695)
0811 06:34:42.585186 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 06:34:42.585191 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:34:42.587213 (+ 2022us) server_negotiation.cc:407] Connection header received
0811 06:34:42.589228 (+ 2015us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:34:42.589235 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:34:42.589387 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:34:42.589505 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:34:42.590918 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:42.591701 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:42.593482 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:42.593978 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:42.594134 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:42.595571 (+ 1437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:34:42.595584 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:34:42.595588 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:34:42.595607 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:34:42.595640 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:34:42.595652 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:34:42.595753 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 06:34:42.596030 (+ 277us) server_negotiation.cc:299] Negotiation successful
0811 06:34:42.596133 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 06:34:42.597743 25474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:42.585901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:44695 (local address 127.28.239.194:35913)
0811 06:34:42.587002 (+ 1101us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:42.587048 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 06:34:42.588927 (+ 1879us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:34:42.589856 (+ 929us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:34:42.589865 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:34:42.589965 (+ 100us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:34:42.590720 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:42.590739 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:42.591871 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:42.591877 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:42.593309 (+ 1432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:42.593324 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:42.594220 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:42.594228 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:42.594475 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:42.595227 (+ 752us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 06:34:42.595229 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:34:42.595271 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 06:34:42.595361 (+ 90us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:34:42.595755 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:34:42.595769 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:34:42.595795 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 06:34:42.596019 (+ 224us) client_negotiation.cc:240] Negotiation successful
0811 06:34:42.596449 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":611,"thread_start_us":378,"threads_started":1}
I20260811 06:34:42.599366 25472 heartbeater.cc:347] Connected to a master server at 127.28.239.254:44695
I20260811 06:34:42.599695 25472 heartbeater.cc:464] Registering TS with master...
I20260811 06:34:42.600481 25472 heartbeater.cc:511] Master 127.28.239.254:44695 requested a full tablet report, sending...
I20260811 06:34:42.602139 24987 ts_manager.cc:194] Registered new tserver with Master: 6162f379978f42859df5c3793466b4df (127.28.239.194:40885)
I20260811 06:34:42.603427 24987 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.28.239.194:35913
W20260811 06:34:42.955006 25475 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:34:42.955348 25475 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:34:42.955437 25475 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:34:42.962694 25475 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:34:42.962803 25475 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:34:42.962908 25475 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.195
I20260811 06:34:42.972049 25475 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:39579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429755927508-29631-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429755927508-29631-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.239.195
--webserver_port=0
--tserver_master_addrs=127.28.239.254:44695
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.195
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429755927508-29631-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:34:42.973366 25475 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:34:42.975220 25475 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:34:42.990206 25484 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:34:42.990231 25489 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:34:42.990188 25485 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:34:42.991262 25475 server_base.cc:1034] running on GCE node
I20260811 06:34:42.991906 25475 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:34:42.992949 25475 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:34:42.994222 25475 hybrid_clock.cc:648] HybridClock initialized: now 1786430082994153 us; error 54 us; skew 500 ppm
I20260811 06:34:42.994526 25475 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:34:43.001312 25475 webserver.cc:460] Webserver started at http://127.28.239.195:46039/ using document root <none> and password file <none>
I20260811 06:34:43.002166 25475 fs_manager.cc:359] Metadata directory not provided
I20260811 06:34:43.002269 25475 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:34:43.002650 25475 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:34:43.005726 25475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429755927508-29631-0/minicluster-data/ts-2/data/instance:
uuid: "91c8cfab99c44236b412453eac468e00"
format_stamp: "Formatted at 2026-08-11 06:34:42 on dist-test-slave-gkw7"
server_key: "bb957e1ade2e820692d36c881911514d"
server_key_iv: "743cea6c8fa4cae0e939561320203084"
server_key_version: "encryptionkey@0"
I20260811 06:34:43.006503 25475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429755927508-29631-0/minicluster-data/ts-2/wal/instance:
uuid: "91c8cfab99c44236b412453eac468e00"
format_stamp: "Formatted at 2026-08-11 06:34:42 on dist-test-slave-gkw7"
server_key: "bb957e1ade2e820692d36c881911514d"
server_key_iv: "743cea6c8fa4cae0e939561320203084"
server_key_version: "encryptionkey@0"
I20260811 06:34:43.011845 25475 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260811 06:34:43.015867 25500 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:34:43.017328 25475 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 06:34:43.017475 25475 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429755927508-29631-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429755927508-29631-0/minicluster-data/ts-2/wal
uuid: "91c8cfab99c44236b412453eac468e00"
format_stamp: "Formatted at 2026-08-11 06:34:42 on dist-test-slave-gkw7"
server_key: "bb957e1ade2e820692d36c881911514d"
server_key_iv: "743cea6c8fa4cae0e939561320203084"
server_key_version: "encryptionkey@0"
I20260811 06:34:43.017761 25475 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429755927508-29631-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429755927508-29631-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429755927508-29631-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:34:43.035209 25475 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:34:43.036044 25475 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:34:43.036319 25475 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:34:43.037348 25475 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:34:43.038897 25475 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:34:43.038995 25475 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:34:43.039075 25475 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:34:43.039132 25475 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:34:43.075063 25475 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.195:37087
I20260811 06:34:43.075125 25686 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.195:37087 every 8 connection(s)
I20260811 06:34:43.076514 25475 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
I20260811 06:34:43.077613 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 25475
I20260811 06:34:43.077868 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429755927508-29631-0/minicluster-data/ts-2/wal/instance
I20260811 06:34:43.078486 29631 external_mini_cluster.cc:1428] Setting key 91bf5430f404a82cb8f946a2333b7b67
I20260811 06:34:43.091728 25272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:43.079746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.195:36057 (local address 127.28.239.254:44695)
0811 06:34:43.079980 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0811 06:34:43.079986 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:34:43.081818 (+ 1832us) server_negotiation.cc:407] Connection header received
0811 06:34:43.083846 (+ 2028us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:34:43.083851 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:34:43.083967 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:34:43.084096 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:34:43.085403 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:43.086251 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:43.088762 (+ 2511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:43.089180 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:43.089284 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:43.090923 (+ 1639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:34:43.090928 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:34:43.090931 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:34:43.090951 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:34:43.090984 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:34:43.090996 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:34:43.091075 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 06:34:43.091370 (+ 295us) server_negotiation.cc:299] Negotiation successful
0811 06:34:43.091455 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260811 06:34:43.093050 25691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:43.080450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:44695 (local address 127.28.239.195:36057)
0811 06:34:43.081633 (+ 1183us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:43.081689 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 06:34:43.083513 (+ 1824us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:34:43.084295 (+ 782us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:34:43.084303 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:34:43.084398 (+ 95us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:34:43.085180 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:43.085198 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:43.086426 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:43.086434 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:43.088590 (+ 2156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:43.088607 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:43.089437 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:43.089446 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:43.089783 (+ 337us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:43.090548 (+ 765us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 06:34:43.090552 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:34:43.090604 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 06:34:43.090710 (+ 106us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:34:43.091105 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:34:43.091117 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:34:43.091126 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 06:34:43.091365 (+ 239us) client_negotiation.cc:240] Negotiation successful
0811 06:34:43.091772 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":575,"thread_start_us":373,"threads_started":1}
I20260811 06:34:43.094488 25687 heartbeater.cc:347] Connected to a master server at 127.28.239.254:44695
I20260811 06:34:43.094852 25687 heartbeater.cc:464] Registering TS with master...
I20260811 06:34:43.095605 25687 heartbeater.cc:511] Master 127.28.239.254:44695 requested a full tablet report, sending...
I20260811 06:34:43.097028 24987 ts_manager.cc:194] Registered new tserver with Master: 91c8cfab99c44236b412453eac468e00 (127.28.239.195:37087)
I20260811 06:34:43.097957 24987 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.28.239.195:36057
I20260811 06:34:43.107470 29631 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 06:34:43.112418 25270 heartbeater.cc:503] Master 127.28.239.254:44695 was elected leader, sending a full tablet report...
I20260811 06:34:43.118295 25272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:43.111919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45984 (local address 127.28.239.254:44695)
0811 06:34:43.112129 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 06:34:43.112135 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:34:43.112606 (+ 471us) server_negotiation.cc:407] Connection header received
0811 06:34:43.112748 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:34:43.112754 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:34:43.112841 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:34:43.112959 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:34:43.113582 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:43.114542 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:43.115991 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:43.116421 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:43.116518 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:43.117579 (+ 1061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:34:43.117588 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:34:43.117596 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:34:43.117625 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:34:43.117659 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:34:43.117672 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:34:43.117745 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 06:34:43.117907 (+ 162us) server_negotiation.cc:299] Negotiation successful
0811 06:34:43.117988 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 06:34:43.122798 24987 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45984:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:34:43.129977 24987 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:34:43.163172 25712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:43.143847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.195:37087 (local address 127.0.0.1:52992)
0811 06:34:43.154584 (+ 10737us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:43.154622 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 06:34:43.154800 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:34:43.155925 (+ 1125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:34:43.155929 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:34:43.155956 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:34:43.156413 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:43.156429 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:43.158072 (+ 1643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:43.158078 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:43.160395 (+ 2317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:43.160408 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:43.162179 (+ 1771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:43.162185 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:43.162398 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:43.162459 (+ 61us) client_negotiation.cc:769] Sending connection context
0811 06:34:43.162588 (+ 129us) client_negotiation.cc:240] Negotiation successful
0811 06:34:43.162784 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10549,"thread_start_us":10348,"threads_started":1}
I20260811 06:34:43.163862 25708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:43.142956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.194:40885 (local address 127.0.0.1:38704)
0811 06:34:43.143813 (+ 857us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:43.143855 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 06:34:43.144050 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:34:43.155697 (+ 11647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:34:43.155709 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:34:43.155742 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:34:43.156273 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:43.156294 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:43.158553 (+ 2259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:43.158558 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:43.161036 (+ 2478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:43.161051 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:43.163021 (+ 1970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:43.163029 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:43.163259 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:43.163295 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 06:34:43.163368 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 06:34:43.163496 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"mutex_wait_us":44,"thread_start_us":351,"threads_started":1}
I20260811 06:34:43.164330 25719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:43.144647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52992 (local address 127.28.239.195:37087)
0811 06:34:43.152055 (+ 7408us) server_negotiation.cc:206] Beginning negotiation
0811 06:34:43.152069 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 06:34:43.154967 (+ 2898us) server_negotiation.cc:407] Connection header received
0811 06:34:43.155101 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:34:43.155109 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:34:43.155394 (+ 285us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:34:43.155621 (+ 227us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:34:43.156674 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:43.157865 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:43.160607 (+ 2742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:43.161957 (+ 1350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:43.163386 (+ 1429us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:43.163630 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0811 06:34:43.163703 (+ 73us) server_negotiation.cc:299] Negotiation successful
0811 06:34:43.163917 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7147,"thread_start_us":6925,"threads_started":1}
I20260811 06:34:43.164652 25715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:43.144201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.193:35535 (local address 127.0.0.1:45932)
0811 06:34:43.155521 (+ 11320us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:43.155541 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 06:34:43.155663 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:34:43.156403 (+ 740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:34:43.156408 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:34:43.156440 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:34:43.156908 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:43.156921 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:43.159105 (+ 2184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:43.159110 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:43.161322 (+ 2212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:43.161333 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:43.163841 (+ 2508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:43.163851 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:43.164066 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:43.164096 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 06:34:43.164191 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 06:34:43.164341 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11182,"thread_start_us":11014,"threads_started":1}
I20260811 06:34:43.164873 25720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:43.144773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45932 (local address 127.28.239.193:35535)
0811 06:34:43.155131 (+ 10358us) server_negotiation.cc:206] Beginning negotiation
0811 06:34:43.155138 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:34:43.155790 (+ 652us) server_negotiation.cc:407] Connection header received
0811 06:34:43.155904 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:34:43.155911 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:34:43.156148 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:34:43.157100 (+ 952us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:34:43.157366 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:43.158932 (+ 1566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:43.161519 (+ 2587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:43.162895 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:43.163091 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:43.163668 (+ 577us) server_negotiation.cc:1035] Waiting for connection context
0811 06:34:43.164283 (+ 615us) server_negotiation.cc:299] Negotiation successful
0811 06:34:43.164475 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10163,"thread_start_us":10067,"threads_started":1}
I20260811 06:34:43.165272 25709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:43.143164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38704 (local address 127.28.239.194:40885)
0811 06:34:43.154907 (+ 11743us) server_negotiation.cc:206] Beginning negotiation
0811 06:34:43.154916 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:34:43.154953 (+ 37us) server_negotiation.cc:407] Connection header received
0811 06:34:43.155070 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:34:43.155081 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:34:43.155403 (+ 322us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:34:43.156522 (+ 1119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:34:43.156768 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:43.158394 (+ 1626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:43.161252 (+ 2858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:43.162781 (+ 1529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:43.163833 (+ 1052us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:43.164282 (+ 449us) server_negotiation.cc:1035] Waiting for connection context
0811 06:34:43.164352 (+ 70us) server_negotiation.cc:299] Negotiation successful
0811 06:34:43.164886 (+ 534us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11564,"thread_start_us":11470,"threads_started":1}
I20260811 06:34:43.168612 25579 tablet_service.cc:1467] Processing CreateTablet for tablet 37188f320d8641e39ff6f41acf81d0ad (DEFAULT_TABLE table=test-table [id=e2517bd5bbca4ce2a420c35aee2d46c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:34:43.168726 25382 tablet_service.cc:1467] Processing CreateTablet for tablet 37188f320d8641e39ff6f41acf81d0ad (DEFAULT_TABLE table=test-table [id=e2517bd5bbca4ce2a420c35aee2d46c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:34:43.168727 25175 tablet_service.cc:1467] Processing CreateTablet for tablet 37188f320d8641e39ff6f41acf81d0ad (DEFAULT_TABLE table=test-table [id=e2517bd5bbca4ce2a420c35aee2d46c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:34:43.170104 25579 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37188f320d8641e39ff6f41acf81d0ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:34:43.170166 25175 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37188f320d8641e39ff6f41acf81d0ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:34:43.170169 25382 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37188f320d8641e39ff6f41acf81d0ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:34:43.182138 25721 tablet_bootstrap.cc:492] T 37188f320d8641e39ff6f41acf81d0ad P 91c8cfab99c44236b412453eac468e00: Bootstrap starting.
I20260811 06:34:43.187435 25721 tablet_bootstrap.cc:654] T 37188f320d8641e39ff6f41acf81d0ad P 91c8cfab99c44236b412453eac468e00: Neither blocks nor log segments found. Creating new log.
I20260811 06:34:43.188586 25721 log.cc:824] T 37188f320d8641e39ff6f41acf81d0ad P 91c8cfab99c44236b412453eac468e00: Log is configured to *not* fsync() on all Append() calls
I20260811 06:34:43.189725 25725 tablet_bootstrap.cc:492] T 37188f320d8641e39ff6f41acf81d0ad P 6162f379978f42859df5c3793466b4df: Bootstrap starting.
I20260811 06:34:43.191591 25726 tablet_bootstrap.cc:492] T 37188f320d8641e39ff6f41acf81d0ad P 2ec144d07b6440eca3fca57e36d4d48e: Bootstrap starting.
I20260811 06:34:43.194700 25725 tablet_bootstrap.cc:654] T 37188f320d8641e39ff6f41acf81d0ad P 6162f379978f42859df5c3793466b4df: Neither blocks nor log segments found. Creating new log.
I20260811 06:34:43.195334 25726 tablet_bootstrap.cc:654] T 37188f320d8641e39ff6f41acf81d0ad P 2ec144d07b6440eca3fca57e36d4d48e: Neither blocks nor log segments found. Creating new log.
I20260811 06:34:43.196156 25721 tablet_bootstrap.cc:492] T 37188f320d8641e39ff6f41acf81d0ad P 91c8cfab99c44236b412453eac468e00: No bootstrap required, opened a new log
I20260811 06:34:43.196147 25725 log.cc:824] T 37188f320d8641e39ff6f41acf81d0ad P 6162f379978f42859df5c3793466b4df: Log is configured to *not* fsync() on all Append() calls
I20260811 06:34:43.196460 25721 ts_tablet_manager.cc:1397] T 37188f320d8641e39ff6f41acf81d0ad P 91c8cfab99c44236b412453eac468e00: Time spent bootstrapping tablet: real 0.015s user 0.001s sys 0.009s
I20260811 06:34:43.196591 25726 log.cc:824] T 37188f320d8641e39ff6f41acf81d0ad P 2ec144d07b6440eca3fca57e36d4d48e: Log is configured to *not* fsync() on all Append() calls
I20260811 06:34:43.199692 25725 tablet_bootstrap.cc:492] T 37188f320d8641e39ff6f41acf81d0ad P 6162f379978f42859df5c3793466b4df: No bootstrap required, opened a new log
I20260811 06:34:43.199947 25726 tablet_bootstrap.cc:492] T 37188f320d8641e39ff6f41acf81d0ad P 2ec144d07b6440eca3fca57e36d4d48e: No bootstrap required, opened a new log
I20260811 06:34:43.200057 25725 ts_tablet_manager.cc:1397] T 37188f320d8641e39ff6f41acf81d0ad P 6162f379978f42859df5c3793466b4df: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260811 06:34:43.200222 25726 ts_tablet_manager.cc:1397] T 37188f320d8641e39ff6f41acf81d0ad P 2ec144d07b6440eca3fca57e36d4d48e: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.006s
I20260811 06:34:43.208173 25725 raft_consensus.cc:348] T 37188f320d8641e39ff6f41acf81d0ad P 6162f379978f42859df5c3793466b4df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6162f379978f42859df5c3793466b4df" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 40885 } } peers { permanent_uuid: "2ec144d07b6440eca3fca57e36d4d48e" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 35535 } } peers { permanent_uuid: "91c8cfab99c44236b412453eac468e00" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 37087 } }
I20260811 06:34:43.208310 25721 raft_consensus.cc:348] T 37188f320d8641e39ff6f41acf81d0ad P 91c8cfab99c44236b412453eac468e00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6162f379978f42859df5c3793466b4df" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 40885 } } peers { permanent_uuid: "2ec144d07b6440eca3fca57e36d4d48e" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 35535 } } peers { permanent_uuid: "91c8cfab99c44236b412453eac468e00" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 37087 } }
I20260811 06:34:43.208643 25725 raft_consensus.cc:374] T 37188f320d8641e39ff6f41acf81d0ad P 6162f379978f42859df5c3793466b4df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:34:43.208757 25725 raft_consensus.cc:729] T 37188f320d8641e39ff6f41acf81d0ad P 6162f379978f42859df5c3793466b4df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6162f379978f42859df5c3793466b4df, State: Initialized, Role: FOLLOWER
I20260811 06:34:43.208786 25721 raft_consensus.cc:374] T 37188f320d8641e39ff6f41acf81d0ad P 91c8cfab99c44236b412453eac468e00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:34:43.208891 25721 raft_consensus.cc:729] T 37188f320d8641e39ff6f41acf81d0ad P 91c8cfab99c44236b412453eac468e00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91c8cfab99c44236b412453eac468e00, State: Initialized, Role: FOLLOWER
I20260811 06:34:43.209465 25725 consensus_queue.cc:260] T 37188f320d8641e39ff6f41acf81d0ad P 6162f379978f42859df5c3793466b4df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6162f379978f42859df5c3793466b4df" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 40885 } } peers { permanent_uuid: "2ec144d07b6440eca3fca57e36d4d48e" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 35535 } } peers { permanent_uuid: "91c8cfab99c44236b412453eac468e00" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 37087 } }
I20260811 06:34:43.209658 25721 consensus_queue.cc:260] T 37188f320d8641e39ff6f41acf81d0ad P 91c8cfab99c44236b412453eac468e00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6162f379978f42859df5c3793466b4df" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 40885 } } peers { permanent_uuid: "2ec144d07b6440eca3fca57e36d4d48e" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 35535 } } peers { permanent_uuid: "91c8cfab99c44236b412453eac468e00" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 37087 } }
I20260811 06:34:43.210909 25472 heartbeater.cc:503] Master 127.28.239.254:44695 was elected leader, sending a full tablet report...
I20260811 06:34:43.211434 25725 ts_tablet_manager.cc:1428] T 37188f320d8641e39ff6f41acf81d0ad P 6162f379978f42859df5c3793466b4df: Time spent starting tablet: real 0.011s user 0.012s sys 0.000s
I20260811 06:34:43.213402 25726 raft_consensus.cc:348] T 37188f320d8641e39ff6f41acf81d0ad P 2ec144d07b6440eca3fca57e36d4d48e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6162f379978f42859df5c3793466b4df" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 40885 } } peers { permanent_uuid: "2ec144d07b6440eca3fca57e36d4d48e" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 35535 } } peers { permanent_uuid: "91c8cfab99c44236b412453eac468e00" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 37087 } }
I20260811 06:34:43.213958 25726 raft_consensus.cc:374] T 37188f320d8641e39ff6f41acf81d0ad P 2ec144d07b6440eca3fca57e36d4d48e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:34:43.214320 25687 heartbeater.cc:503] Master 127.28.239.254:44695 was elected leader, sending a full tablet report...
I20260811 06:34:43.215070 25721 ts_tablet_manager.cc:1428] T 37188f320d8641e39ff6f41acf81d0ad P 91c8cfab99c44236b412453eac468e00: Time spent starting tablet: real 0.018s user 0.018s sys 0.000s
I20260811 06:34:43.214057 25726 raft_consensus.cc:729] T 37188f320d8641e39ff6f41acf81d0ad P 2ec144d07b6440eca3fca57e36d4d48e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ec144d07b6440eca3fca57e36d4d48e, State: Initialized, Role: FOLLOWER
I20260811 06:34:43.216017 25726 consensus_queue.cc:260] T 37188f320d8641e39ff6f41acf81d0ad P 2ec144d07b6440eca3fca57e36d4d48e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6162f379978f42859df5c3793466b4df" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 40885 } } peers { permanent_uuid: "2ec144d07b6440eca3fca57e36d4d48e" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 35535 } } peers { permanent_uuid: "91c8cfab99c44236b412453eac468e00" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 37087 } }
I20260811 06:34:43.217973 25726 ts_tablet_manager.cc:1428] T 37188f320d8641e39ff6f41acf81d0ad P 2ec144d07b6440eca3fca57e36d4d48e: Time spent starting tablet: real 0.018s user 0.014s sys 0.003s
I20260811 06:34:43.268246 25736 raft_consensus.cc:482] T 37188f320d8641e39ff6f41acf81d0ad P 91c8cfab99c44236b412453eac468e00 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:34:43.268584 25736 raft_consensus.cc:504] T 37188f320d8641e39ff6f41acf81d0ad P 91c8cfab99c44236b412453eac468e00 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6162f379978f42859df5c3793466b4df" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 40885 } } peers { permanent_uuid: "2ec144d07b6440eca3fca57e36d4d48e" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 35535 } } peers { permanent_uuid: "91c8cfab99c44236b412453eac468e00" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 37087 } }
I20260811 06:34:43.270452 25736 leader_election.cc:288] T 37188f320d8641e39ff6f41acf81d0ad P 91c8cfab99c44236b412453eac468e00 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6162f379978f42859df5c3793466b4df (127.28.239.194:40885), 2ec144d07b6440eca3fca57e36d4d48e (127.28.239.193:35535)
I20260811 06:34:43.278777 25709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:43.271100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.195:51067 (local address 127.28.239.194:40885)
0811 06:34:43.271367 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0811 06:34:43.271376 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:34:43.271441 (+ 65us) server_negotiation.cc:407] Connection header received
0811 06:34:43.271531 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:34:43.271538 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:34:43.271672 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:34:43.271785 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:34:43.272350 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:43.273493 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:43.276385 (+ 2892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:43.277625 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:43.277801 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:43.277896 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 06:34:43.278337 (+ 441us) server_negotiation.cc:299] Negotiation successful
0811 06:34:43.278478 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260811 06:34:43.279292 25691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:43.270985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.194:40885 (local address 127.28.239.195:51067)
0811 06:34:43.271219 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:43.271239 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 06:34:43.271372 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:34:43.271857 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:34:43.271862 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:34:43.271898 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:34:43.272144 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:43.272161 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:43.273718 (+ 1557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:43.273725 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:43.276163 (+ 2438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:43.276178 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:43.277801 (+ 1623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:43.277808 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:43.278112 (+ 304us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:43.278151 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 06:34:43.278266 (+ 115us) client_negotiation.cc:240] Negotiation successful
0811 06:34:43.278431 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71}
I20260811 06:34:43.280745 25720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:43.271904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.195:33827 (local address 127.28.239.193:35535)
0811 06:34:43.272127 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0811 06:34:43.272133 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:34:43.272674 (+ 541us) server_negotiation.cc:407] Connection header received
0811 06:34:43.272899 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:34:43.272905 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:34:43.272999 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:34:43.273089 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:34:43.274010 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:43.275440 (+ 1430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:43.278066 (+ 2626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:43.279345 (+ 1279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:43.279529 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:43.279600 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 06:34:43.280345 (+ 745us) server_negotiation.cc:299] Negotiation successful
0811 06:34:43.280494 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260811 06:34:43.281162 25412 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37188f320d8641e39ff6f41acf81d0ad" candidate_uuid: "91c8cfab99c44236b412453eac468e00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6162f379978f42859df5c3793466b4df" is_pre_election: true
I20260811 06:34:43.281471 25739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:43.271551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.193:35535 (local address 127.28.239.195:33827)
0811 06:34:43.272539 (+ 988us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:43.272572 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0811 06:34:43.272726 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:34:43.273245 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:34:43.273249 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:34:43.273273 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:34:43.273689 (+ 416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:43.273705 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:43.275648 (+ 1943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:43.275657 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:43.277757 (+ 2100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:43.277785 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:43.279914 (+ 2129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:43.279922 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:43.280125 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:43.280160 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 06:34:43.281002 (+ 842us) client_negotiation.cc:240] Negotiation successful
0811 06:34:43.281146 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":862,"thread_start_us":595,"threads_started":1}
I20260811 06:34:43.281992 25412 raft_consensus.cc:2454] T 37188f320d8641e39ff6f41acf81d0ad P 6162f379978f42859df5c3793466b4df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91c8cfab99c44236b412453eac468e00 in term 0.
I20260811 06:34:43.282883 25503 leader_election.cc:302] T 37188f320d8641e39ff6f41acf81d0ad P 91c8cfab99c44236b412453eac468e00 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6162f379978f42859df5c3793466b4df, 91c8cfab99c44236b412453eac468e00; no voters:
I20260811 06:34:43.282812 25212 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37188f320d8641e39ff6f41acf81d0ad" candidate_uuid: "91c8cfab99c44236b412453eac468e00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ec144d07b6440eca3fca57e36d4d48e" is_pre_election: true
I20260811 06:34:43.283386 25736 raft_consensus.cc:2781] T 37188f320d8641e39ff6f41acf81d0ad P 91c8cfab99c44236b412453eac468e00 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:34:43.283555 25736 raft_consensus.cc:482] T 37188f320d8641e39ff6f41acf81d0ad P 91c8cfab99c44236b412453eac468e00 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:34:43.283509 25212 raft_consensus.cc:2454] T 37188f320d8641e39ff6f41acf81d0ad P 2ec144d07b6440eca3fca57e36d4d48e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91c8cfab99c44236b412453eac468e00 in term 0.
I20260811 06:34:43.283663 25736 raft_consensus.cc:3037] T 37188f320d8641e39ff6f41acf81d0ad P 91c8cfab99c44236b412453eac468e00 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:34:43.286298 25736 raft_consensus.cc:504] T 37188f320d8641e39ff6f41acf81d0ad P 91c8cfab99c44236b412453eac468e00 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6162f379978f42859df5c3793466b4df" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 40885 } } peers { permanent_uuid: "2ec144d07b6440eca3fca57e36d4d48e" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 35535 } } peers { permanent_uuid: "91c8cfab99c44236b412453eac468e00" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 37087 } }
I20260811 06:34:43.286978 25736 leader_election.cc:288] T 37188f320d8641e39ff6f41acf81d0ad P 91c8cfab99c44236b412453eac468e00 [CANDIDATE]: Term 1 election: Requested vote from peers 6162f379978f42859df5c3793466b4df (127.28.239.194:40885), 2ec144d07b6440eca3fca57e36d4d48e (127.28.239.193:35535)
I20260811 06:34:43.287519 25212 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37188f320d8641e39ff6f41acf81d0ad" candidate_uuid: "91c8cfab99c44236b412453eac468e00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ec144d07b6440eca3fca57e36d4d48e"
I20260811 06:34:43.287515 25412 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37188f320d8641e39ff6f41acf81d0ad" candidate_uuid: "91c8cfab99c44236b412453eac468e00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6162f379978f42859df5c3793466b4df"
I20260811 06:34:43.287748 25212 raft_consensus.cc:3037] T 37188f320d8641e39ff6f41acf81d0ad P 2ec144d07b6440eca3fca57e36d4d48e [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:34:43.287758 25412 raft_consensus.cc:3037] T 37188f320d8641e39ff6f41acf81d0ad P 6162f379978f42859df5c3793466b4df [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:34:43.290164 25212 raft_consensus.cc:2454] T 37188f320d8641e39ff6f41acf81d0ad P 2ec144d07b6440eca3fca57e36d4d48e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91c8cfab99c44236b412453eac468e00 in term 1.
I20260811 06:34:43.290423 25412 raft_consensus.cc:2454] T 37188f320d8641e39ff6f41acf81d0ad P 6162f379978f42859df5c3793466b4df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91c8cfab99c44236b412453eac468e00 in term 1.
I20260811 06:34:43.290701 25503 leader_election.cc:302] T 37188f320d8641e39ff6f41acf81d0ad P 91c8cfab99c44236b412453eac468e00 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ec144d07b6440eca3fca57e36d4d48e, 91c8cfab99c44236b412453eac468e00; no voters:
I20260811 06:34:43.291018 25736 raft_consensus.cc:2781] T 37188f320d8641e39ff6f41acf81d0ad P 91c8cfab99c44236b412453eac468e00 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:34:43.291512 25736 raft_consensus.cc:686] T 37188f320d8641e39ff6f41acf81d0ad P 91c8cfab99c44236b412453eac468e00 [term 1 LEADER]: Becoming Leader. State: Replica: 91c8cfab99c44236b412453eac468e00, State: Running, Role: LEADER
I20260811 06:34:43.292079 25736 consensus_queue.cc:237] T 37188f320d8641e39ff6f41acf81d0ad P 91c8cfab99c44236b412453eac468e00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6162f379978f42859df5c3793466b4df" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 40885 } } peers { permanent_uuid: "2ec144d07b6440eca3fca57e36d4d48e" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 35535 } } peers { permanent_uuid: "91c8cfab99c44236b412453eac468e00" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 37087 } }
I20260811 06:34:43.297730 24987 catalog_manager.cc:5495] T 37188f320d8641e39ff6f41acf81d0ad P 91c8cfab99c44236b412453eac468e00 reported cstate change: term changed from 0 to 1, leader changed from <none> to 91c8cfab99c44236b412453eac468e00 (127.28.239.195). New cstate: current_term: 1 leader_uuid: "91c8cfab99c44236b412453eac468e00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6162f379978f42859df5c3793466b4df" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 40885 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ec144d07b6440eca3fca57e36d4d48e" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 35535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91c8cfab99c44236b412453eac468e00" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 37087 } health_report { overall_health: HEALTHY } } }
I20260811 06:34:43.325160 25719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:43.318762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53008 (local address 127.28.239.195:37087)
0811 06:34:43.319004 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0811 06:34:43.319011 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:34:43.319034 (+ 23us) server_negotiation.cc:407] Connection header received
0811 06:34:43.319153 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:34:43.319157 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:34:43.319238 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:34:43.319424 (+ 186us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:34:43.319820 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:43.320565 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:43.322568 (+ 2003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:43.323071 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:43.323240 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:43.323862 (+ 622us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:34:43.324409 (+ 547us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:34:43.324616 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0811 06:34:43.324729 (+ 113us) server_negotiation.cc:299] Negotiation successful
0811 06:34:43.324822 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
W20260811 06:34:43.328346 25689 tablet.cc:2369] T 37188f320d8641e39ff6f41acf81d0ad P 91c8cfab99c44236b412453eac468e00: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:34:43.334390 25412 raft_consensus.cc:1261] T 37188f320d8641e39ff6f41acf81d0ad P 6162f379978f42859df5c3793466b4df [term 1 FOLLOWER]: Refusing update from remote peer 91c8cfab99c44236b412453eac468e00: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:34:43.334388 25212 raft_consensus.cc:1261] T 37188f320d8641e39ff6f41acf81d0ad P 2ec144d07b6440eca3fca57e36d4d48e [term 1 FOLLOWER]: Refusing update from remote peer 91c8cfab99c44236b412453eac468e00: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260811 06:34:43.335023 25473 tablet.cc:2369] T 37188f320d8641e39ff6f41acf81d0ad P 6162f379978f42859df5c3793466b4df: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:34:43.335453 25740 consensus_queue.cc:1035] T 37188f320d8641e39ff6f41acf81d0ad P 91c8cfab99c44236b412453eac468e00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ec144d07b6440eca3fca57e36d4d48e" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 35535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:34:43.335909 25736 consensus_queue.cc:1035] T 37188f320d8641e39ff6f41acf81d0ad P 91c8cfab99c44236b412453eac468e00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6162f379978f42859df5c3793466b4df" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 40885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 06:34:43.340260 25271 tablet.cc:2369] T 37188f320d8641e39ff6f41acf81d0ad P 2ec144d07b6440eca3fca57e36d4d48e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:34:43.347256 25743 mvcc.cc:205] Tried to move back new op lower bound from 7317217621326868480 to 7317217621168328704. Current Snapshot: MvccSnapshot[applied={T|T < 7317217621326868480}]
I20260811 06:34:43.362816 24986 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45984:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:34:43.363067 24986 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45984:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:34:43.366844 24986 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 54e72c9836f947a1a31517960671cfeb: Sending DeleteTablet for 3 replicas of tablet 37188f320d8641e39ff6f41acf81d0ad
I20260811 06:34:43.368023 25382 tablet_service.cc:1514] Processing DeleteTablet for tablet 37188f320d8641e39ff6f41acf81d0ad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:34:43 UTC) from {username='slave'} at 127.0.0.1:38704
I20260811 06:34:43.368114 25579 tablet_service.cc:1514] Processing DeleteTablet for tablet 37188f320d8641e39ff6f41acf81d0ad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:34:43 UTC) from {username='slave'} at 127.0.0.1:52992
I20260811 06:34:43.368916 25175 tablet_service.cc:1514] Processing DeleteTablet for tablet 37188f320d8641e39ff6f41acf81d0ad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:34:43 UTC) from {username='slave'} at 127.0.0.1:45932
I20260811 06:34:43.370395 25757 tablet_replica.cc:330] T 37188f320d8641e39ff6f41acf81d0ad P 6162f379978f42859df5c3793466b4df: stopping tablet replica
I20260811 06:34:43.370769 25759 tablet_replica.cc:330] T 37188f320d8641e39ff6f41acf81d0ad P 2ec144d07b6440eca3fca57e36d4d48e: stopping tablet replica
I20260811 06:34:43.371023 25757 raft_consensus.cc:2229] T 37188f320d8641e39ff6f41acf81d0ad P 6162f379978f42859df5c3793466b4df [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:34:43.371217 25759 raft_consensus.cc:2229] T 37188f320d8641e39ff6f41acf81d0ad P 2ec144d07b6440eca3fca57e36d4d48e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:34:43.371694 25757 raft_consensus.cc:2258] T 37188f320d8641e39ff6f41acf81d0ad P 6162f379978f42859df5c3793466b4df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:34:43.371733 25759 raft_consensus.cc:2258] T 37188f320d8641e39ff6f41acf81d0ad P 2ec144d07b6440eca3fca57e36d4d48e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:34:43.371889 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 25073
I20260811 06:34:43.372545 25758 tablet_replica.cc:330] T 37188f320d8641e39ff6f41acf81d0ad P 91c8cfab99c44236b412453eac468e00: stopping tablet replica
I20260811 06:34:43.372953 25758 raft_consensus.cc:2229] T 37188f320d8641e39ff6f41acf81d0ad P 91c8cfab99c44236b412453eac468e00 [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:34:43.373709 25758 raft_consensus.cc:2258] T 37188f320d8641e39ff6f41acf81d0ad P 91c8cfab99c44236b412453eac468e00 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:34:43.374823 25757 ts_tablet_manager.cc:1905] T 37188f320d8641e39ff6f41acf81d0ad P 6162f379978f42859df5c3793466b4df: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:34:43.376223 25758 ts_tablet_manager.cc:1905] T 37188f320d8641e39ff6f41acf81d0ad P 91c8cfab99c44236b412453eac468e00: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:34:43.382278 25757 ts_tablet_manager.cc:1918] T 37188f320d8641e39ff6f41acf81d0ad P 6162f379978f42859df5c3793466b4df: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:34:43.382567 25757 log.cc:1196] T 37188f320d8641e39ff6f41acf81d0ad P 6162f379978f42859df5c3793466b4df: Deleting WAL directory at /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429755927508-29631-0/minicluster-data/ts-1/wal/wals/37188f320d8641e39ff6f41acf81d0ad
I20260811 06:34:43.383308 25759 ts_tablet_manager.cc:1905] T 37188f320d8641e39ff6f41acf81d0ad P 2ec144d07b6440eca3fca57e36d4d48e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:34:43.383350 25757 ts_tablet_manager.cc:1939] T 37188f320d8641e39ff6f41acf81d0ad P 6162f379978f42859df5c3793466b4df: Deleting consensus metadata
I20260811 06:34:43.384680 25758 ts_tablet_manager.cc:1918] T 37188f320d8641e39ff6f41acf81d0ad P 91c8cfab99c44236b412453eac468e00: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:34:43.384876 25758 log.cc:1196] T 37188f320d8641e39ff6f41acf81d0ad P 91c8cfab99c44236b412453eac468e00: Deleting WAL directory at /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429755927508-29631-0/minicluster-data/ts-2/wal/wals/37188f320d8641e39ff6f41acf81d0ad
I20260811 06:34:43.385473 25758 ts_tablet_manager.cc:1939] T 37188f320d8641e39ff6f41acf81d0ad P 91c8cfab99c44236b412453eac468e00: Deleting consensus metadata
I20260811 06:34:43.386018 24969 catalog_manager.cc:4841] TS 6162f379978f42859df5c3793466b4df (127.28.239.194:40885): tablet 37188f320d8641e39ff6f41acf81d0ad (table test-table [id=e2517bd5bbca4ce2a420c35aee2d46c9]) successfully deleted
I20260811 06:34:43.387136 24968 catalog_manager.cc:4841] TS 91c8cfab99c44236b412453eac468e00 (127.28.239.195:37087): tablet 37188f320d8641e39ff6f41acf81d0ad (table test-table [id=e2517bd5bbca4ce2a420c35aee2d46c9]) successfully deleted
I20260811 06:34:43.391492 25759 ts_tablet_manager.cc:1918] T 37188f320d8641e39ff6f41acf81d0ad P 2ec144d07b6440eca3fca57e36d4d48e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:34:43.391714 25759 log.cc:1196] T 37188f320d8641e39ff6f41acf81d0ad P 2ec144d07b6440eca3fca57e36d4d48e: Deleting WAL directory at /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786429755927508-29631-0/minicluster-data/ts-0/wal/wals/37188f320d8641e39ff6f41acf81d0ad
I20260811 06:34:43.392318 25759 ts_tablet_manager.cc:1939] T 37188f320d8641e39ff6f41acf81d0ad P 2ec144d07b6440eca3fca57e36d4d48e: Deleting consensus metadata
I20260811 06:34:43.394143 24969 catalog_manager.cc:4841] TS 2ec144d07b6440eca3fca57e36d4d48e (127.28.239.193:35535): tablet 37188f320d8641e39ff6f41acf81d0ad (table test-table [id=e2517bd5bbca4ce2a420c35aee2d46c9]) successfully deleted
I20260811 06:34:43.395946 25720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:43.374052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45938 (local address 127.28.239.193:35535)
0811 06:34:43.374454 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0811 06:34:43.374460 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:34:43.387759 (+ 13299us) server_negotiation.cc:407] Connection header received
0811 06:34:43.387967 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:34:43.387974 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:34:43.388072 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:34:43.388179 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:34:43.388644 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:43.389941 (+ 1297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:43.391984 (+ 2043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:43.392740 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:43.392883 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:43.394205 (+ 1322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:34:43.394218 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:34:43.394236 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:34:43.394921 (+ 685us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:34:43.395003 (+ 82us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:34:43.395029 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:34:43.395128 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 06:34:43.395328 (+ 200us) server_negotiation.cc:299] Negotiation successful
0811 06:34:43.395469 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":255}
I20260811 06:34:43.396754 25265 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:34:43.504520 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 25073
I20260811 06:34:43.534689 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 25278
I20260811 06:34:43.542896 25709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:43.535608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38708 (local address 127.28.239.194:40885)
0811 06:34:43.535839 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0811 06:34:43.535846 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:34:43.535871 (+ 25us) server_negotiation.cc:407] Connection header received
0811 06:34:43.535993 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:34:43.536000 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:34:43.536128 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:34:43.536255 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:34:43.536646 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:43.537868 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:43.539165 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:43.539818 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:43.539948 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:43.541098 (+ 1150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:34:43.541110 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:34:43.541130 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:34:43.541968 (+ 838us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:34:43.542029 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:34:43.542059 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:34:43.542183 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 06:34:43.542366 (+ 183us) server_negotiation.cc:299] Negotiation successful
0811 06:34:43.542487 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 06:34:43.543699 25464 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:34:43.637894 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 25278
I20260811 06:34:43.675105 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 25475
I20260811 06:34:43.683019 25719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:43.676032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53018 (local address 127.28.239.195:37087)
0811 06:34:43.676222 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 06:34:43.676229 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:34:43.676311 (+ 82us) server_negotiation.cc:407] Connection header received
0811 06:34:43.676552 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:34:43.676556 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:34:43.676656 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:34:43.676792 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:34:43.677232 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:43.678140 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:43.679475 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:43.679950 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:43.680100 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:43.681284 (+ 1184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:34:43.681295 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:34:43.681312 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:34:43.682074 (+ 762us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:34:43.682125 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:34:43.682148 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:34:43.682293 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 06:34:43.682427 (+ 134us) server_negotiation.cc:299] Negotiation successful
0811 06:34:43.682554 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 06:34:43.683724 25679 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:34:43.814967 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 25475
I20260811 06:34:43.843881 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 24940
I20260811 06:34:43.845166 25029 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:34:43.958439 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 24940
2026-08-11T06:34:43Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (3116 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:34:44 dist-test-slave-gkw7 krb5kdc[25773](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:34:44 dist-test-slave-gkw7 krb5kdc[25773](info): set up 2 sockets
Aug 11 06:34:44 dist-test-slave-gkw7 krb5kdc[25773](info): commencing operation
krb5kdc: starting...
W20260811 06:34:45.339108 29631 mini_kdc.cc:121] Time spent starting KDC: real 1.329s user 0.000s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/krb5kdc/test-user.keytab.
Aug 11 06:34:45 dist-test-slave-gkw7 krb5kdc[25773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430085, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:34:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:34:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.239.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.239.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.239.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab.
Entry for principal kudu/127.28.239.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab.
I20260811 06:34:45.666923 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:34637
--webserver_interface=127.28.239.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab
--principal=kudu/127.28.239.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:44651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.239.254:34637
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:34:46.026381 25788 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:34:46.026747 25788 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:34:46.026926 25788 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:34:46.034279 25788 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:34:46.034435 25788 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:34:46.034548 25788 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:34:46.034647 25788 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:34:46.034736 25788 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:34:46.044823 25788 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:44651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.239.254:34637
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab
--principal=kudu/127.28.239.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:34637
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.239.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:34:46.046300 25788 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:34:46.047691 25788 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:34:46.060101 25796 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:34:46.060133 25799 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:34:46.060248 25795 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:34:46.061370 25788 server_base.cc:1034] running on GCE node
I20260811 06:34:46.062238 25788 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:34:46.063793 25788 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:34:46.065088 25788 hybrid_clock.cc:648] HybridClock initialized: now 1786430086065024 us; error 42 us; skew 500 ppm
I20260811 06:34:46.065642 25788 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:34:46 dist-test-slave-gkw7 krb5kdc[25773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430086, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:34:46.069025 25788 init.cc:377] Logged in from keytab as kudu/127.28.239.254@KRBTEST.COM (short username kudu)
I20260811 06:34:46.070683 25788 webserver.cc:460] Webserver started at http://127.28.239.254:35677/ using document root <none> and password file <none>
I20260811 06:34:46.071449 25788 fs_manager.cc:359] Metadata directory not provided
I20260811 06:34:46.071599 25788 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:34:46.071956 25788 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:34:46.075229 25788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/minicluster-data/master-0/data/instance:
uuid: "8c46112b50dc480fad0a4432e1ee7d99"
format_stamp: "Formatted at 2026-08-11 06:34:46 on dist-test-slave-gkw7"
server_key: "6f930fd21b280e6f09e0f6ae4dc93f1b"
server_key_iv: "2a33c4f367a11b9d2ac93019b7b09b12"
server_key_version: "encryptionkey@0"
I20260811 06:34:46.076035 25788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/minicluster-data/master-0/wal/instance:
uuid: "8c46112b50dc480fad0a4432e1ee7d99"
format_stamp: "Formatted at 2026-08-11 06:34:46 on dist-test-slave-gkw7"
server_key: "6f930fd21b280e6f09e0f6ae4dc93f1b"
server_key_iv: "2a33c4f367a11b9d2ac93019b7b09b12"
server_key_version: "encryptionkey@0"
I20260811 06:34:46.081676 25788 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260811 06:34:46.085661 25808 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:34:46.087173 25788 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:34:46.087378 25788 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/minicluster-data/master-0/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/minicluster-data/master-0/wal
uuid: "8c46112b50dc480fad0a4432e1ee7d99"
format_stamp: "Formatted at 2026-08-11 06:34:46 on dist-test-slave-gkw7"
server_key: "6f930fd21b280e6f09e0f6ae4dc93f1b"
server_key_iv: "2a33c4f367a11b9d2ac93019b7b09b12"
server_key_version: "encryptionkey@0"
I20260811 06:34:46.087608 25788 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:34:46.102828 25788 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:34:46.107609 25788 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:34:46.107944 25788 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:34:46.128531 25788 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.254:34637
I20260811 06:34:46.128573 25888 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.254:34637 every 8 connection(s)
I20260811 06:34:46.130061 25788 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
I20260811 06:34:46.134202 25890 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:34:46.140264 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 25788
I20260811 06:34:46.140470 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/minicluster-data/master-0/wal/instance
I20260811 06:34:46.141057 29631 external_mini_cluster.cc:1428] Setting key 45b925f83102244523cadc8467e31531
I20260811 06:34:46.146355 25890 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c46112b50dc480fad0a4432e1ee7d99: Bootstrap starting.
I20260811 06:34:46.149643 25890 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8c46112b50dc480fad0a4432e1ee7d99: Neither blocks nor log segments found. Creating new log.
I20260811 06:34:46.150786 25890 log.cc:824] T 00000000000000000000000000000000 P 8c46112b50dc480fad0a4432e1ee7d99: Log is configured to *not* fsync() on all Append() calls
I20260811 06:34:46.153949 25890 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c46112b50dc480fad0a4432e1ee7d99: No bootstrap required, opened a new log
Aug 11 06:34:46 dist-test-slave-gkw7 krb5kdc[25773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430085, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:34:46.162662 25890 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8c46112b50dc480fad0a4432e1ee7d99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c46112b50dc480fad0a4432e1ee7d99" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 34637 } }
I20260811 06:34:46.163053 25890 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8c46112b50dc480fad0a4432e1ee7d99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:34:46.163264 25890 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8c46112b50dc480fad0a4432e1ee7d99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c46112b50dc480fad0a4432e1ee7d99, State: Initialized, Role: FOLLOWER
I20260811 06:34:46.163965 25890 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8c46112b50dc480fad0a4432e1ee7d99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c46112b50dc480fad0a4432e1ee7d99" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 34637 } }
I20260811 06:34:46.164270 25890 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8c46112b50dc480fad0a4432e1ee7d99 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:34:46.164426 25890 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8c46112b50dc480fad0a4432e1ee7d99 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:34:46.164606 25890 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8c46112b50dc480fad0a4432e1ee7d99 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:34:46.167229 25890 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8c46112b50dc480fad0a4432e1ee7d99 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c46112b50dc480fad0a4432e1ee7d99" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 34637 } }
I20260811 06:34:46.167819 25890 leader_election.cc:302] T 00000000000000000000000000000000 P 8c46112b50dc480fad0a4432e1ee7d99 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8c46112b50dc480fad0a4432e1ee7d99; no voters:
I20260811 06:34:46.168533 25890 leader_election.cc:288] T 00000000000000000000000000000000 P 8c46112b50dc480fad0a4432e1ee7d99 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:34:46.168766 25896 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8c46112b50dc480fad0a4432e1ee7d99 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:34:46.169098 25894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:46.143354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57642 (local address 127.28.239.254:34637)
0811 06:34:46.144299 (+ 945us) server_negotiation.cc:206] Beginning negotiation
0811 06:34:46.144316 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 06:34:46.144363 (+ 47us) server_negotiation.cc:407] Connection header received
0811 06:34:46.146362 (+ 1999us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:34:46.146388 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:34:46.146889 (+ 501us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:34:46.147322 (+ 433us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:34:46.149040 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:46.150663 (+ 1623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:46.152695 (+ 2032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:46.153468 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:46.153734 (+ 266us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:46.158889 (+ 5155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:34:46.158898 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:34:46.158921 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:34:46.158967 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:34:46.162976 (+ 4009us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:34:46.164054 (+ 1078us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:34:46.164064 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:34:46.164072 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:34:46.164384 (+ 312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:34:46.164960 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:34:46.164968 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:34:46.164973 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:34:46.165857 (+ 884us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:34:46.166281 (+ 424us) server_negotiation.cc:1035] Waiting for connection context
0811 06:34:46.166847 (+ 566us) server_negotiation.cc:299] Negotiation successful
0811 06:34:46.167390 (+ 543us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":661,"thread_start_us":402,"threads_started":1}
I20260811 06:34:46.169905 25896 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8c46112b50dc480fad0a4432e1ee7d99 [term 1 LEADER]: Becoming Leader. State: Replica: 8c46112b50dc480fad0a4432e1ee7d99, State: Running, Role: LEADER
I20260811 06:34:46.170694 25896 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8c46112b50dc480fad0a4432e1ee7d99 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c46112b50dc480fad0a4432e1ee7d99" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 34637 } }
I20260811 06:34:46.170948 25890 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8c46112b50dc480fad0a4432e1ee7d99 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:34:46.173971 25897 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c46112b50dc480fad0a4432e1ee7d99 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8c46112b50dc480fad0a4432e1ee7d99" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c46112b50dc480fad0a4432e1ee7d99" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 34637 } } }
I20260811 06:34:46.174458 25897 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c46112b50dc480fad0a4432e1ee7d99 [sys.catalog]: This master's current role is: LEADER
I20260811 06:34:46.175587 25903 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:34:46.175419 25898 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c46112b50dc480fad0a4432e1ee7d99 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8c46112b50dc480fad0a4432e1ee7d99. Latest consensus state: current_term: 1 leader_uuid: "8c46112b50dc480fad0a4432e1ee7d99" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c46112b50dc480fad0a4432e1ee7d99" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 34637 } } }
I20260811 06:34:46.176239 25898 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c46112b50dc480fad0a4432e1ee7d99 [sys.catalog]: This master's current role is: LEADER
I20260811 06:34:46.184201 25903 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:34:46.194785 25903 catalog_manager.cc:1347] Generated new cluster ID: 4e89a7de422b404f979139f143bd697d
I20260811 06:34:46.194964 25903 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:34:46.234800 25903 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:34:46.237066 25903 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:34:46.255622 25903 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8c46112b50dc480fad0a4432e1ee7d99: Generated new TSK 0
I20260811 06:34:46.256702 25903 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.239.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.239.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.239.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.193.keytab.
Entry for principal kudu/127.28.239.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.193.keytab.
I20260811 06:34:46.359472 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.193:0
--local_ip_for_outbound_sockets=127.28.239.193
--webserver_interface=127.28.239.193
--webserver_port=0
--tserver_master_addrs=127.28.239.254:34637
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.193.keytab
--principal=kudu/127.28.239.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:44651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:34:46.736025 25925 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:34:46.736323 25925 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:34:46.736418 25925 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:34:46.743713 25925 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:34:46.743850 25925 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:34:46.743983 25925 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.193
I20260811 06:34:46.753911 25925 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:44651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.193.keytab
--principal=kudu/127.28.239.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.239.193
--webserver_port=0
--tserver_master_addrs=127.28.239.254:34637
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.193
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:34:46.755357 25925 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:34:46.756764 25925 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:34:46.770275 25930 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:34:46.770238 25933 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:34:46.770321 25931 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:34:46.771699 25925 server_base.cc:1034] running on GCE node
I20260811 06:34:46.772341 25925 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:34:46.773258 25925 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:34:46.774485 25925 hybrid_clock.cc:648] HybridClock initialized: now 1786430086774401 us; error 71 us; skew 500 ppm
I20260811 06:34:46.774838 25925 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:34:46 dist-test-slave-gkw7 krb5kdc[25773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430086, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:34:46.777972 25925 init.cc:377] Logged in from keytab as kudu/127.28.239.193@KRBTEST.COM (short username kudu)
I20260811 06:34:46.779583 25925 webserver.cc:460] Webserver started at http://127.28.239.193:34515/ using document root <none> and password file <none>
I20260811 06:34:46.780316 25925 fs_manager.cc:359] Metadata directory not provided
I20260811 06:34:46.780472 25925 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:34:46.780812 25925 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:34:46.783881 25925 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/minicluster-data/ts-0/data/instance:
uuid: "3f72ffdf95d148b9a12768960ec220b5"
format_stamp: "Formatted at 2026-08-11 06:34:46 on dist-test-slave-gkw7"
server_key: "5bc3a0839a349bf1c001bd93dd078909"
server_key_iv: "91733d6ffba3ee4448712d4c8dea69fe"
server_key_version: "encryptionkey@0"
I20260811 06:34:46.784693 25925 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/minicluster-data/ts-0/wal/instance:
uuid: "3f72ffdf95d148b9a12768960ec220b5"
format_stamp: "Formatted at 2026-08-11 06:34:46 on dist-test-slave-gkw7"
server_key: "5bc3a0839a349bf1c001bd93dd078909"
server_key_iv: "91733d6ffba3ee4448712d4c8dea69fe"
server_key_version: "encryptionkey@0"
I20260811 06:34:46.790098 25925 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 06:34:46.794008 25949 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:34:46.795444 25925 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260811 06:34:46.795588 25925 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/minicluster-data/ts-0/wal
uuid: "3f72ffdf95d148b9a12768960ec220b5"
format_stamp: "Formatted at 2026-08-11 06:34:46 on dist-test-slave-gkw7"
server_key: "5bc3a0839a349bf1c001bd93dd078909"
server_key_iv: "91733d6ffba3ee4448712d4c8dea69fe"
server_key_version: "encryptionkey@0"
I20260811 06:34:46.795760 25925 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:34:46.844046 25925 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:34:46.848912 25925 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:34:46.849179 25925 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:34:46.850282 25925 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:34:46.851645 25925 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:34:46.851722 25925 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:34:46.851811 25925 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:34:46.851857 25925 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:34:46.887584 25925 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.193:34259
I20260811 06:34:46.887641 26109 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.193:34259 every 8 connection(s)
I20260811 06:34:46.889057 25925 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
I20260811 06:34:46.896920 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 25925
I20260811 06:34:46.897222 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/minicluster-data/ts-0/wal/instance
I20260811 06:34:46.897800 29631 external_mini_cluster.cc:1428] Setting key 71e98aa9b01eb1dbea2b97b9f72da323
Aug 11 06:34:46 dist-test-slave-gkw7 krb5kdc[25773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430086, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.193@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:34:46.912106 26112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:46.892162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.193:57743 (local address 127.28.239.254:34637)
0811 06:34:46.892750 (+ 588us) server_negotiation.cc:206] Beginning negotiation
0811 06:34:46.892755 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:34:46.894029 (+ 1274us) server_negotiation.cc:407] Connection header received
0811 06:34:46.896000 (+ 1971us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:34:46.896015 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:34:46.896126 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:34:46.896250 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:34:46.898357 (+ 2107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:46.899448 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:46.901484 (+ 2036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:46.902010 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:46.902155 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:46.906082 (+ 3927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:34:46.906087 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:34:46.906091 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:34:46.906125 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:34:46.908712 (+ 2587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:34:46.909506 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:34:46.909511 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:34:46.909515 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:34:46.909700 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:34:46.910172 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:34:46.910177 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:34:46.910180 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:34:46.910540 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:34:46.910704 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 06:34:46.911471 (+ 767us) server_negotiation.cc:299] Negotiation successful
0811 06:34:46.911716 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":461,"thread_start_us":361,"threads_started":1}
I20260811 06:34:46.913218 26117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:46.893023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:34637 (local address 127.28.239.193:57743)
0811 06:34:46.893856 (+ 833us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:46.893897 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 06:34:46.895661 (+ 1764us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:34:46.896570 (+ 909us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:34:46.896580 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:34:46.897292 (+ 712us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:34:46.898127 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:46.898149 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:46.899626 (+ 1477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:46.899631 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:46.901317 (+ 1686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:46.901332 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:46.902291 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:46.902300 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:46.902559 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:46.903412 (+ 853us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:34:46.903444 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:34:46.905822 (+ 2378us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:34:46.908888 (+ 3066us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:34:46.908897 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:34:46.908915 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:34:46.909366 (+ 451us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:34:46.909828 (+ 462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:34:46.909833 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:34:46.909837 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:34:46.910042 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:34:46.910692 (+ 650us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:34:46.910702 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:34:46.911061 (+ 359us) client_negotiation.cc:769] Sending connection context
0811 06:34:46.911360 (+ 299us) client_negotiation.cc:240] Negotiation successful
0811 06:34:46.911929 (+ 569us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":297,"threads_started":1}
I20260811 06:34:46.915037 26110 heartbeater.cc:347] Connected to a master server at 127.28.239.254:34637
I20260811 06:34:46.915392 26110 heartbeater.cc:464] Registering TS with master...
I20260811 06:34:46.916311 26110 heartbeater.cc:511] Master 127.28.239.254:34637 requested a full tablet report, sending...
I20260811 06:34:46.918680 25838 ts_manager.cc:194] Registered new tserver with Master: 3f72ffdf95d148b9a12768960ec220b5 (127.28.239.193:34259)
I20260811 06:34:46.920159 25838 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.239.193@KRBTEST.COM'} at 127.28.239.193:57743
WARNING: no policy specified for kudu/127.28.239.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.239.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.239.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.194.keytab.
Entry for principal kudu/127.28.239.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.194.keytab.
I20260811 06:34:46.971009 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.194:0
--local_ip_for_outbound_sockets=127.28.239.194
--webserver_interface=127.28.239.194
--webserver_port=0
--tserver_master_addrs=127.28.239.254:34637
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.194.keytab
--principal=kudu/127.28.239.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:44651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:34:47.338819 26123 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:34:47.339138 26123 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:34:47.339236 26123 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:34:47.346527 26123 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:34:47.346642 26123 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:34:47.346765 26123 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.194
I20260811 06:34:47.356606 26123 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:44651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.194.keytab
--principal=kudu/127.28.239.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.239.194
--webserver_port=0
--tserver_master_addrs=127.28.239.254:34637
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.194
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:34:47.357921 26123 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:34:47.359277 26123 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:34:47.373450 26136 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:34:47.373559 26132 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:34:47.373512 26134 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:34:47.374459 26123 server_base.cc:1034] running on GCE node
I20260811 06:34:47.375154 26123 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:34:47.376157 26123 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:34:47.377390 26123 hybrid_clock.cc:648] HybridClock initialized: now 1786430087377344 us; error 61 us; skew 500 ppm
I20260811 06:34:47.377787 26123 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:34:47 dist-test-slave-gkw7 krb5kdc[25773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430087, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:34:47.381264 26123 init.cc:377] Logged in from keytab as kudu/127.28.239.194@KRBTEST.COM (short username kudu)
I20260811 06:34:47.382910 26123 webserver.cc:460] Webserver started at http://127.28.239.194:37693/ using document root <none> and password file <none>
I20260811 06:34:47.383638 26123 fs_manager.cc:359] Metadata directory not provided
I20260811 06:34:47.383785 26123 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:34:47.384130 26123 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:34:47.387224 26123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/minicluster-data/ts-1/data/instance:
uuid: "6c20a80fb808443795647957db66a896"
format_stamp: "Formatted at 2026-08-11 06:34:47 on dist-test-slave-gkw7"
server_key: "b175cb5b3e41ffbe82529e2485d1a4c4"
server_key_iv: "c462607bb8f695571fe905a1146ff834"
server_key_version: "encryptionkey@0"
I20260811 06:34:47.388041 26123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/minicluster-data/ts-1/wal/instance:
uuid: "6c20a80fb808443795647957db66a896"
format_stamp: "Formatted at 2026-08-11 06:34:47 on dist-test-slave-gkw7"
server_key: "b175cb5b3e41ffbe82529e2485d1a4c4"
server_key_iv: "c462607bb8f695571fe905a1146ff834"
server_key_version: "encryptionkey@0"
I20260811 06:34:47.393710 26123 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260811 06:34:47.397292 26144 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:34:47.398774 26123 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:34:47.398957 26123 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/minicluster-data/ts-1/wal
uuid: "6c20a80fb808443795647957db66a896"
format_stamp: "Formatted at 2026-08-11 06:34:47 on dist-test-slave-gkw7"
server_key: "b175cb5b3e41ffbe82529e2485d1a4c4"
server_key_iv: "c462607bb8f695571fe905a1146ff834"
server_key_version: "encryptionkey@0"
I20260811 06:34:47.399185 26123 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:34:47.434352 26123 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:34:47.439142 26123 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:34:47.439456 26123 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:34:47.440517 26123 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:34:47.441998 26123 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:34:47.442111 26123 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:34:47.442269 26123 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:34:47.442342 26123 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:34:47.477033 26123 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.194:43319
I20260811 06:34:47.477092 26310 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.194:43319 every 8 connection(s)
I20260811 06:34:47.478523 26123 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
I20260811 06:34:47.487088 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 26123
I20260811 06:34:47.487317 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/minicluster-data/ts-1/wal/instance
I20260811 06:34:47.487815 29631 external_mini_cluster.cc:1428] Setting key 9b5fe171146bd594a878b40eaffb8eee
Aug 11 06:34:47 dist-test-slave-gkw7 krb5kdc[25773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430087, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.194@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:34:47.500162 26314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:47.481243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.194:54783 (local address 127.28.239.254:34637)
0811 06:34:47.481749 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0811 06:34:47.481754 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:34:47.482785 (+ 1031us) server_negotiation.cc:407] Connection header received
0811 06:34:47.484685 (+ 1900us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:34:47.484691 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:34:47.484794 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:34:47.484923 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:34:47.486590 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:47.487422 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:47.489266 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:47.489919 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:47.490060 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:47.494141 (+ 4081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:34:47.494146 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:34:47.494150 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:34:47.494182 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:34:47.496853 (+ 2671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:34:47.497669 (+ 816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:34:47.497677 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:34:47.497685 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:34:47.497814 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:34:47.498294 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:34:47.498298 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:34:47.498300 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:34:47.498601 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:34:47.498792 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0811 06:34:47.499505 (+ 713us) server_negotiation.cc:299] Negotiation successful
0811 06:34:47.499737 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":398,"thread_start_us":271,"threads_started":1}
I20260811 06:34:47.501245 26315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:47.481923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:34637 (local address 127.28.239.194:54783)
0811 06:34:47.482622 (+ 699us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:47.482663 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 06:34:47.484402 (+ 1739us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:34:47.485187 (+ 785us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:34:47.485196 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:34:47.485829 (+ 633us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:34:47.486408 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:47.486426 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:47.487592 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:47.487596 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:47.489116 (+ 1520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:47.489129 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:47.490169 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:47.490177 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:47.490540 (+ 363us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:47.491421 (+ 881us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:34:47.491449 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:34:47.493894 (+ 2445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:34:47.497009 (+ 3115us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:34:47.497019 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:34:47.497037 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:34:47.497487 (+ 450us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:34:47.497943 (+ 456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:34:47.497947 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:34:47.497951 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:34:47.498151 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:34:47.498742 (+ 591us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:34:47.498750 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:34:47.499131 (+ 381us) client_negotiation.cc:769] Sending connection context
0811 06:34:47.499412 (+ 281us) client_negotiation.cc:240] Negotiation successful
0811 06:34:47.500073 (+ 661us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":277,"threads_started":1}
I20260811 06:34:47.502988 26311 heartbeater.cc:347] Connected to a master server at 127.28.239.254:34637
I20260811 06:34:47.503342 26311 heartbeater.cc:464] Registering TS with master...
I20260811 06:34:47.504256 26311 heartbeater.cc:511] Master 127.28.239.254:34637 requested a full tablet report, sending...
I20260811 06:34:47.505991 25838 ts_manager.cc:194] Registered new tserver with Master: 6c20a80fb808443795647957db66a896 (127.28.239.194:43319)
I20260811 06:34:47.506843 25838 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.239.194@KRBTEST.COM'} at 127.28.239.194:54783
WARNING: no policy specified for kudu/127.28.239.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.239.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.239.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.195.keytab.
Entry for principal kudu/127.28.239.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.195.keytab.
I20260811 06:34:47.561350 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.195:0
--local_ip_for_outbound_sockets=127.28.239.195
--webserver_interface=127.28.239.195
--webserver_port=0
--tserver_master_addrs=127.28.239.254:34637
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.195.keytab
--principal=kudu/127.28.239.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:44651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:34:47.917364 26318 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:34:47.917749 26318 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:34:47.917960 26318 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260811 06:34:47.924075 26110 heartbeater.cc:503] Master 127.28.239.254:34637 was elected leader, sending a full tablet report...
W20260811 06:34:47.926225 26318 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:34:47.926373 26318 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:34:47.926523 26318 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.195
I20260811 06:34:47.936654 26318 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:44651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.195.keytab
--principal=kudu/127.28.239.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.239.195
--webserver_port=0
--tserver_master_addrs=127.28.239.254:34637
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.195
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:34:47.938071 26318 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:34:47.939385 26318 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:34:47.952986 26323 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:34:47.952941 26328 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:34:47.953027 26324 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:34:47.954461 26318 server_base.cc:1034] running on GCE node
I20260811 06:34:47.955135 26318 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:34:47.956151 26318 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:34:47.957396 26318 hybrid_clock.cc:648] HybridClock initialized: now 1786430087957333 us; error 56 us; skew 500 ppm
I20260811 06:34:47.957787 26318 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:34:47 dist-test-slave-gkw7 krb5kdc[25773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430087, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:34:47.961031 26318 init.cc:377] Logged in from keytab as kudu/127.28.239.195@KRBTEST.COM (short username kudu)
I20260811 06:34:47.962610 26318 webserver.cc:460] Webserver started at http://127.28.239.195:46735/ using document root <none> and password file <none>
I20260811 06:34:47.963299 26318 fs_manager.cc:359] Metadata directory not provided
I20260811 06:34:47.963403 26318 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:34:47.963709 26318 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:34:47.966987 26318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/minicluster-data/ts-2/data/instance:
uuid: "7fb26319cf7544d4b19a81765d7e245a"
format_stamp: "Formatted at 2026-08-11 06:34:47 on dist-test-slave-gkw7"
server_key: "213681d89883fec6507740cca55db85a"
server_key_iv: "387142666e0da13425e4d00c819accd6"
server_key_version: "encryptionkey@0"
I20260811 06:34:47.967869 26318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/minicluster-data/ts-2/wal/instance:
uuid: "7fb26319cf7544d4b19a81765d7e245a"
format_stamp: "Formatted at 2026-08-11 06:34:47 on dist-test-slave-gkw7"
server_key: "213681d89883fec6507740cca55db85a"
server_key_iv: "387142666e0da13425e4d00c819accd6"
server_key_version: "encryptionkey@0"
I20260811 06:34:47.973471 26318 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 06:34:47.977106 26338 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:34:47.978582 26318 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:34:47.978798 26318 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/minicluster-data/ts-2/wal
uuid: "7fb26319cf7544d4b19a81765d7e245a"
format_stamp: "Formatted at 2026-08-11 06:34:47 on dist-test-slave-gkw7"
server_key: "213681d89883fec6507740cca55db85a"
server_key_iv: "387142666e0da13425e4d00c819accd6"
server_key_version: "encryptionkey@0"
I20260811 06:34:47.978986 26318 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:34:48.010442 26318 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:34:48.015097 26318 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:34:48.015374 26318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:34:48.016402 26318 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:34:48.017904 26318 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:34:48.017984 26318 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:34:48.018070 26318 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:34:48.018116 26318 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:34:48.054719 26318 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.195:39829
I20260811 06:34:48.054833 26507 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.195:39829 every 8 connection(s)
I20260811 06:34:48.056255 26318 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
I20260811 06:34:48.066251 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 26318
I20260811 06:34:48.066496 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/minicluster-data/ts-2/wal/instance
I20260811 06:34:48.066994 29631 external_mini_cluster.cc:1428] Setting key 0b1cabf2b2a9d4ec7a5d6ae68f779270
Aug 11 06:34:48 dist-test-slave-gkw7 krb5kdc[25773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430087, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.195@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:34:48.078429 26513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:48.059250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.195:56091 (local address 127.28.239.254:34637)
0811 06:34:48.059708 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0811 06:34:48.059712 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:34:48.060840 (+ 1128us) server_negotiation.cc:407] Connection header received
0811 06:34:48.062866 (+ 2026us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:34:48.062872 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:34:48.062976 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:34:48.063101 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:34:48.064819 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:48.065669 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:48.067428 (+ 1759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:48.067852 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:48.068005 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:48.072114 (+ 4109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:34:48.072119 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:34:48.072122 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:34:48.072157 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:34:48.075038 (+ 2881us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:34:48.075834 (+ 796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:34:48.075840 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:34:48.075845 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:34:48.075982 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:34:48.076455 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:34:48.076462 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:34:48.076465 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:34:48.076767 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:34:48.076952 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0811 06:34:48.077730 (+ 778us) server_negotiation.cc:299] Negotiation successful
0811 06:34:48.077990 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":222,"threads_started":1}
I20260811 06:34:48.079526 26514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:48.059901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:34637 (local address 127.28.239.195:56091)
0811 06:34:48.060651 (+ 750us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:48.060692 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 06:34:48.062571 (+ 1879us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:34:48.063358 (+ 787us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:34:48.063367 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:34:48.064011 (+ 644us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:34:48.064625 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:48.064643 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:48.065851 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:48.065855 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:48.067262 (+ 1407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:48.067288 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:48.068101 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:48.068108 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:48.068362 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:48.069178 (+ 816us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:34:48.069204 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:34:48.071898 (+ 2694us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:34:48.075195 (+ 3297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:34:48.075205 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:34:48.075223 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:34:48.075698 (+ 475us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:34:48.076104 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:34:48.076110 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:34:48.076113 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:34:48.076336 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:34:48.076915 (+ 579us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:34:48.076924 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:34:48.077310 (+ 386us) client_negotiation.cc:769] Sending connection context
0811 06:34:48.077645 (+ 335us) client_negotiation.cc:240] Negotiation successful
0811 06:34:48.078270 (+ 625us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":305,"threads_started":1}
I20260811 06:34:48.081283 26510 heartbeater.cc:347] Connected to a master server at 127.28.239.254:34637
I20260811 06:34:48.081707 26510 heartbeater.cc:464] Registering TS with master...
I20260811 06:34:48.082700 26510 heartbeater.cc:511] Master 127.28.239.254:34637 requested a full tablet report, sending...
I20260811 06:34:48.084308 25838 ts_manager.cc:194] Registered new tserver with Master: 7fb26319cf7544d4b19a81765d7e245a (127.28.239.195:39829)
I20260811 06:34:48.085135 25838 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.239.195@KRBTEST.COM'} at 127.28.239.195:56091
I20260811 06:34:48.095942 29631 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 06:34:48.115571 26513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:48.100601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60386 (local address 127.28.239.254:34637)
0811 06:34:48.100837 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0811 06:34:48.100842 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:34:48.101361 (+ 519us) server_negotiation.cc:407] Connection header received
0811 06:34:48.101506 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:34:48.101510 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:34:48.101647 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:34:48.101735 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:34:48.103587 (+ 1852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:48.104356 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:48.105966 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:48.106489 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:48.106601 (+ 112us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:48.109943 (+ 3342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:34:48.109948 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:34:48.109951 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:34:48.109986 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:34:48.112608 (+ 2622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:34:48.113562 (+ 954us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:34:48.113565 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:34:48.113567 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:34:48.113750 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:34:48.114241 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:34:48.114244 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:34:48.114246 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:34:48.114595 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:34:48.114810 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0811 06:34:48.115162 (+ 352us) server_negotiation.cc:299] Negotiation successful
0811 06:34:48.115329 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260811 06:34:48.119843 25838 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60386:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:34:48.127008 25838 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:34:48.162220 26531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:48.141497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55592 (local address 127.28.239.194:43319)
0811 06:34:48.154579 (+ 13082us) server_negotiation.cc:206] Beginning negotiation
0811 06:34:48.154589 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:34:48.154619 (+ 30us) server_negotiation.cc:407] Connection header received
0811 06:34:48.154748 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:34:48.154761 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:34:48.155037 (+ 276us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:34:48.155283 (+ 246us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:34:48.156093 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:48.157244 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:48.159886 (+ 2642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:48.160872 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:48.161069 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:48.161288 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0811 06:34:48.161671 (+ 383us) server_negotiation.cc:299] Negotiation successful
0811 06:34:48.161878 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12909,"thread_start_us":12303,"threads_started":1}
I20260811 06:34:48.162266 26528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:48.141207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.194:43319 (local address 127.0.0.1:55592)
0811 06:34:48.143336 (+ 2129us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:48.143400 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0811 06:34:48.143606 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:34:48.155294 (+ 11688us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:34:48.155302 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:34:48.155332 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:34:48.155872 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:48.155886 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:48.157468 (+ 1582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:48.157473 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:48.159703 (+ 2230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:48.159717 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:48.161128 (+ 1411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:48.161134 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:48.161334 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:48.161425 (+ 91us) client_negotiation.cc:769] Sending connection context
0811 06:34:48.161608 (+ 183us) client_negotiation.cc:240] Negotiation successful
0811 06:34:48.161842 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1791,"mutex_wait_us":55,"thread_start_us":1606,"threads_started":1}
I20260811 06:34:48.164218 26530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:48.141776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55260 (local address 127.28.239.193:34259)
0811 06:34:48.142612 (+ 836us) server_negotiation.cc:206] Beginning negotiation
0811 06:34:48.142623 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 06:34:48.153270 (+ 10647us) server_negotiation.cc:407] Connection header received
0811 06:34:48.154778 (+ 1508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:34:48.154789 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:34:48.155064 (+ 275us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:34:48.155295 (+ 231us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:34:48.156340 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:48.158110 (+ 1770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:48.161417 (+ 3307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:48.162983 (+ 1566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:48.163209 (+ 226us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:48.163395 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0811 06:34:48.163666 (+ 271us) server_negotiation.cc:299] Negotiation successful
0811 06:34:48.163842 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":564,"thread_start_us":410,"threads_started":1}
I20260811 06:34:48.164431 26535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:48.143909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.195:39829 (local address 127.0.0.1:37720)
0811 06:34:48.154680 (+ 10771us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:48.154699 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 06:34:48.154827 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:34:48.155614 (+ 787us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:34:48.155618 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:34:48.155643 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:34:48.156195 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:48.156214 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:48.158297 (+ 2083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:48.158301 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:48.161184 (+ 2883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:48.161199 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:48.163538 (+ 2339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:48.163547 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:48.163722 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:48.163754 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 06:34:48.163846 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 06:34:48.163966 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10632,"mutex_wait_us":134,"thread_start_us":10483,"threads_started":1}
I20260811 06:34:48.165000 26536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:48.144168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37720 (local address 127.28.239.195:39829)
0811 06:34:48.154209 (+ 10041us) server_negotiation.cc:206] Beginning negotiation
0811 06:34:48.154220 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 06:34:48.154825 (+ 605us) server_negotiation.cc:407] Connection header received
0811 06:34:48.154948 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:34:48.154954 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:34:48.155144 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:34:48.155311 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:34:48.156438 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:48.158086 (+ 1648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:48.161385 (+ 3299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:48.163005 (+ 1620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:48.163225 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:48.163391 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0811 06:34:48.164622 (+ 1231us) server_negotiation.cc:299] Negotiation successful
0811 06:34:48.164763 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9851,"thread_start_us":9682,"threads_started":1}
I20260811 06:34:48.165035 26529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:48.141214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.193:34259 (local address 127.0.0.1:55260)
0811 06:34:48.153081 (+ 11867us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:48.153110 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0811 06:34:48.154121 (+ 1011us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:34:48.155569 (+ 1448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:34:48.155574 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:34:48.155607 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:34:48.156162 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:48.156172 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:48.158287 (+ 2115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:48.158293 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:48.161186 (+ 2893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:48.161199 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:48.163205 (+ 2006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:48.163211 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:48.163396 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:48.163439 (+ 43us) client_negotiation.cc:769] Sending connection context
0811 06:34:48.164524 (+ 1085us) client_negotiation.cc:240] Negotiation successful
0811 06:34:48.164674 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11678,"mutex_wait_us":561,"thread_start_us":11802,"threads_started":1}
I20260811 06:34:48.167338 26212 tablet_service.cc:1467] Processing CreateTablet for tablet 3ccb168c9a844cdc9377a3ce7e89b1ef (DEFAULT_TABLE table=test-table [id=eb330727b39648f6982f6fce878c0828]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:34:48.169459 26212 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ccb168c9a844cdc9377a3ce7e89b1ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:34:48.169020 26022 tablet_service.cc:1467] Processing CreateTablet for tablet 3ccb168c9a844cdc9377a3ce7e89b1ef (DEFAULT_TABLE table=test-table [id=eb330727b39648f6982f6fce878c0828]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:34:48.169019 26407 tablet_service.cc:1467] Processing CreateTablet for tablet 3ccb168c9a844cdc9377a3ce7e89b1ef (DEFAULT_TABLE table=test-table [id=eb330727b39648f6982f6fce878c0828]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:34:48.170490 26022 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ccb168c9a844cdc9377a3ce7e89b1ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:34:48.170507 26407 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ccb168c9a844cdc9377a3ce7e89b1ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:34:48.181855 26538 tablet_bootstrap.cc:492] T 3ccb168c9a844cdc9377a3ce7e89b1ef P 6c20a80fb808443795647957db66a896: Bootstrap starting.
I20260811 06:34:48.185976 26538 tablet_bootstrap.cc:654] T 3ccb168c9a844cdc9377a3ce7e89b1ef P 6c20a80fb808443795647957db66a896: Neither blocks nor log segments found. Creating new log.
I20260811 06:34:48.187141 26538 log.cc:824] T 3ccb168c9a844cdc9377a3ce7e89b1ef P 6c20a80fb808443795647957db66a896: Log is configured to *not* fsync() on all Append() calls
I20260811 06:34:48.190402 26540 tablet_bootstrap.cc:492] T 3ccb168c9a844cdc9377a3ce7e89b1ef P 7fb26319cf7544d4b19a81765d7e245a: Bootstrap starting.
I20260811 06:34:48.190644 26538 tablet_bootstrap.cc:492] T 3ccb168c9a844cdc9377a3ce7e89b1ef P 6c20a80fb808443795647957db66a896: No bootstrap required, opened a new log
I20260811 06:34:48.190723 26539 tablet_bootstrap.cc:492] T 3ccb168c9a844cdc9377a3ce7e89b1ef P 3f72ffdf95d148b9a12768960ec220b5: Bootstrap starting.
I20260811 06:34:48.191159 26538 ts_tablet_manager.cc:1397] T 3ccb168c9a844cdc9377a3ce7e89b1ef P 6c20a80fb808443795647957db66a896: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.006s
I20260811 06:34:48.194839 26539 tablet_bootstrap.cc:654] T 3ccb168c9a844cdc9377a3ce7e89b1ef P 3f72ffdf95d148b9a12768960ec220b5: Neither blocks nor log segments found. Creating new log.
I20260811 06:34:48.196234 26539 log.cc:824] T 3ccb168c9a844cdc9377a3ce7e89b1ef P 3f72ffdf95d148b9a12768960ec220b5: Log is configured to *not* fsync() on all Append() calls
I20260811 06:34:48.197023 26540 tablet_bootstrap.cc:654] T 3ccb168c9a844cdc9377a3ce7e89b1ef P 7fb26319cf7544d4b19a81765d7e245a: Neither blocks nor log segments found. Creating new log.
I20260811 06:34:48.198496 26540 log.cc:824] T 3ccb168c9a844cdc9377a3ce7e89b1ef P 7fb26319cf7544d4b19a81765d7e245a: Log is configured to *not* fsync() on all Append() calls
I20260811 06:34:48.200115 26539 tablet_bootstrap.cc:492] T 3ccb168c9a844cdc9377a3ce7e89b1ef P 3f72ffdf95d148b9a12768960ec220b5: No bootstrap required, opened a new log
I20260811 06:34:48.200430 26539 ts_tablet_manager.cc:1397] T 3ccb168c9a844cdc9377a3ce7e89b1ef P 3f72ffdf95d148b9a12768960ec220b5: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.000s
I20260811 06:34:48.201438 26540 tablet_bootstrap.cc:492] T 3ccb168c9a844cdc9377a3ce7e89b1ef P 7fb26319cf7544d4b19a81765d7e245a: No bootstrap required, opened a new log
I20260811 06:34:48.201741 26540 ts_tablet_manager.cc:1397] T 3ccb168c9a844cdc9377a3ce7e89b1ef P 7fb26319cf7544d4b19a81765d7e245a: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.006s
I20260811 06:34:48.202049 26538 raft_consensus.cc:348] T 3ccb168c9a844cdc9377a3ce7e89b1ef P 6c20a80fb808443795647957db66a896 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c20a80fb808443795647957db66a896" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 43319 } } peers { permanent_uuid: "3f72ffdf95d148b9a12768960ec220b5" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 34259 } } peers { permanent_uuid: "7fb26319cf7544d4b19a81765d7e245a" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 39829 } }
I20260811 06:34:48.202589 26538 raft_consensus.cc:374] T 3ccb168c9a844cdc9377a3ce7e89b1ef P 6c20a80fb808443795647957db66a896 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:34:48.202702 26538 raft_consensus.cc:729] T 3ccb168c9a844cdc9377a3ce7e89b1ef P 6c20a80fb808443795647957db66a896 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c20a80fb808443795647957db66a896, State: Initialized, Role: FOLLOWER
I20260811 06:34:48.203393 26538 consensus_queue.cc:260] T 3ccb168c9a844cdc9377a3ce7e89b1ef P 6c20a80fb808443795647957db66a896 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c20a80fb808443795647957db66a896" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 43319 } } peers { permanent_uuid: "3f72ffdf95d148b9a12768960ec220b5" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 34259 } } peers { permanent_uuid: "7fb26319cf7544d4b19a81765d7e245a" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 39829 } }
I20260811 06:34:48.204636 26311 heartbeater.cc:503] Master 127.28.239.254:34637 was elected leader, sending a full tablet report...
I20260811 06:34:48.205317 26538 ts_tablet_manager.cc:1428] T 3ccb168c9a844cdc9377a3ce7e89b1ef P 6c20a80fb808443795647957db66a896: Time spent starting tablet: real 0.014s user 0.009s sys 0.005s
I20260811 06:34:48.210035 26539 raft_consensus.cc:348] T 3ccb168c9a844cdc9377a3ce7e89b1ef P 3f72ffdf95d148b9a12768960ec220b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c20a80fb808443795647957db66a896" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 43319 } } peers { permanent_uuid: "3f72ffdf95d148b9a12768960ec220b5" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 34259 } } peers { permanent_uuid: "7fb26319cf7544d4b19a81765d7e245a" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 39829 } }
I20260811 06:34:48.210593 26539 raft_consensus.cc:374] T 3ccb168c9a844cdc9377a3ce7e89b1ef P 3f72ffdf95d148b9a12768960ec220b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:34:48.210718 26539 raft_consensus.cc:729] T 3ccb168c9a844cdc9377a3ce7e89b1ef P 3f72ffdf95d148b9a12768960ec220b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f72ffdf95d148b9a12768960ec220b5, State: Initialized, Role: FOLLOWER
I20260811 06:34:48.211356 26539 consensus_queue.cc:260] T 3ccb168c9a844cdc9377a3ce7e89b1ef P 3f72ffdf95d148b9a12768960ec220b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c20a80fb808443795647957db66a896" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 43319 } } peers { permanent_uuid: "3f72ffdf95d148b9a12768960ec220b5" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 34259 } } peers { permanent_uuid: "7fb26319cf7544d4b19a81765d7e245a" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 39829 } }
I20260811 06:34:48.213053 26539 ts_tablet_manager.cc:1428] T 3ccb168c9a844cdc9377a3ce7e89b1ef P 3f72ffdf95d148b9a12768960ec220b5: Time spent starting tablet: real 0.013s user 0.003s sys 0.011s
I20260811 06:34:48.212149 26540 raft_consensus.cc:348] T 3ccb168c9a844cdc9377a3ce7e89b1ef P 7fb26319cf7544d4b19a81765d7e245a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c20a80fb808443795647957db66a896" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 43319 } } peers { permanent_uuid: "3f72ffdf95d148b9a12768960ec220b5" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 34259 } } peers { permanent_uuid: "7fb26319cf7544d4b19a81765d7e245a" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 39829 } }
I20260811 06:34:48.213719 26540 raft_consensus.cc:374] T 3ccb168c9a844cdc9377a3ce7e89b1ef P 7fb26319cf7544d4b19a81765d7e245a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:34:48.213848 26540 raft_consensus.cc:729] T 3ccb168c9a844cdc9377a3ce7e89b1ef P 7fb26319cf7544d4b19a81765d7e245a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fb26319cf7544d4b19a81765d7e245a, State: Initialized, Role: FOLLOWER
I20260811 06:34:48.214562 26540 consensus_queue.cc:260] T 3ccb168c9a844cdc9377a3ce7e89b1ef P 7fb26319cf7544d4b19a81765d7e245a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c20a80fb808443795647957db66a896" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 43319 } } peers { permanent_uuid: "3f72ffdf95d148b9a12768960ec220b5" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 34259 } } peers { permanent_uuid: "7fb26319cf7544d4b19a81765d7e245a" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 39829 } }
I20260811 06:34:48.215986 26510 heartbeater.cc:503] Master 127.28.239.254:34637 was elected leader, sending a full tablet report...
I20260811 06:34:48.216586 26540 ts_tablet_manager.cc:1428] T 3ccb168c9a844cdc9377a3ce7e89b1ef P 7fb26319cf7544d4b19a81765d7e245a: Time spent starting tablet: real 0.015s user 0.006s sys 0.007s
I20260811 06:34:48.221360 26546 raft_consensus.cc:482] T 3ccb168c9a844cdc9377a3ce7e89b1ef P 7fb26319cf7544d4b19a81765d7e245a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:34:48.221626 26546 raft_consensus.cc:504] T 3ccb168c9a844cdc9377a3ce7e89b1ef P 7fb26319cf7544d4b19a81765d7e245a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c20a80fb808443795647957db66a896" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 43319 } } peers { permanent_uuid: "3f72ffdf95d148b9a12768960ec220b5" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 34259 } } peers { permanent_uuid: "7fb26319cf7544d4b19a81765d7e245a" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 39829 } }
I20260811 06:34:48.223193 26546 leader_election.cc:288] T 3ccb168c9a844cdc9377a3ce7e89b1ef P 7fb26319cf7544d4b19a81765d7e245a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c20a80fb808443795647957db66a896 (127.28.239.194:43319), 3f72ffdf95d148b9a12768960ec220b5 (127.28.239.193:34259)
I20260811 06:34:48.229192 26531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:48.223708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.195:42539 (local address 127.28.239.194:43319)
0811 06:34:48.223917 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 06:34:48.223923 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:34:48.223946 (+ 23us) server_negotiation.cc:407] Connection header received
0811 06:34:48.224036 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:34:48.224044 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:34:48.224131 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:34:48.224247 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:34:48.224656 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:48.225614 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:48.227461 (+ 1847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:48.228275 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:48.228445 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:48.228522 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 06:34:48.228810 (+ 288us) server_negotiation.cc:299] Negotiation successful
0811 06:34:48.228939 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 06:34:48.229198 26514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:48.223448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.194:43319 (local address 127.28.239.195:42539)
0811 06:34:48.223750 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:48.223772 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 06:34:48.223908 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:34:48.224280 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:34:48.224291 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:34:48.224318 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:34:48.224492 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:48.224500 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:48.225777 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:48.225782 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:48.227295 (+ 1513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:48.227311 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:48.228437 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:48.228443 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:48.228633 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:48.228657 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 06:34:48.228761 (+ 104us) client_negotiation.cc:240] Negotiation successful
0811 06:34:48.228898 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142}
I20260811 06:34:48.229895 26530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:48.223707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.195:54413 (local address 127.28.239.193:34259)
0811 06:34:48.223917 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 06:34:48.223923 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:34:48.224538 (+ 615us) server_negotiation.cc:407] Connection header received
0811 06:34:48.224733 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:34:48.224738 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:34:48.224796 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:34:48.224894 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:34:48.225427 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:48.226464 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:48.228147 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:48.228928 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:48.229112 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:48.229193 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 06:34:48.229502 (+ 309us) server_negotiation.cc:299] Negotiation successful
0811 06:34:48.229653 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 06:34:48.229962 26548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:48.223540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.193:34259 (local address 127.28.239.195:54413)
0811 06:34:48.224422 (+ 882us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:48.224439 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 06:34:48.224598 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:34:48.224909 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:34:48.224914 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:34:48.224937 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:34:48.225253 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:48.225262 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:48.226628 (+ 1366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:48.226634 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:48.227984 (+ 1350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:48.227996 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:48.229132 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:48.229138 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:48.229338 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:48.229387 (+ 49us) client_negotiation.cc:769] Sending connection context
0811 06:34:48.229476 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 06:34:48.229644 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":746,"thread_start_us":560,"threads_started":1}
W20260811 06:34:48.230749 26313 tablet.cc:2369] T 3ccb168c9a844cdc9377a3ce7e89b1ef P 6c20a80fb808443795647957db66a896: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:34:48.230732 26246 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ccb168c9a844cdc9377a3ce7e89b1ef" candidate_uuid: "7fb26319cf7544d4b19a81765d7e245a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c20a80fb808443795647957db66a896" is_pre_election: true
I20260811 06:34:48.230985 26047 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ccb168c9a844cdc9377a3ce7e89b1ef" candidate_uuid: "7fb26319cf7544d4b19a81765d7e245a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f72ffdf95d148b9a12768960ec220b5" is_pre_election: true
I20260811 06:34:48.231406 26246 raft_consensus.cc:2454] T 3ccb168c9a844cdc9377a3ce7e89b1ef P 6c20a80fb808443795647957db66a896 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fb26319cf7544d4b19a81765d7e245a in term 0.
I20260811 06:34:48.231539 26047 raft_consensus.cc:2454] T 3ccb168c9a844cdc9377a3ce7e89b1ef P 3f72ffdf95d148b9a12768960ec220b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fb26319cf7544d4b19a81765d7e245a in term 0.
I20260811 06:34:48.232154 26342 leader_election.cc:302] T 3ccb168c9a844cdc9377a3ce7e89b1ef P 7fb26319cf7544d4b19a81765d7e245a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 3f72ffdf95d148b9a12768960ec220b5, 6c20a80fb808443795647957db66a896, 7fb26319cf7544d4b19a81765d7e245a; no voters:
I20260811 06:34:48.232568 26546 raft_consensus.cc:2781] T 3ccb168c9a844cdc9377a3ce7e89b1ef P 7fb26319cf7544d4b19a81765d7e245a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:34:48.232724 26546 raft_consensus.cc:482] T 3ccb168c9a844cdc9377a3ce7e89b1ef P 7fb26319cf7544d4b19a81765d7e245a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:34:48.232827 26546 raft_consensus.cc:3037] T 3ccb168c9a844cdc9377a3ce7e89b1ef P 7fb26319cf7544d4b19a81765d7e245a [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:34:48.235191 26546 raft_consensus.cc:504] T 3ccb168c9a844cdc9377a3ce7e89b1ef P 7fb26319cf7544d4b19a81765d7e245a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c20a80fb808443795647957db66a896" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 43319 } } peers { permanent_uuid: "3f72ffdf95d148b9a12768960ec220b5" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 34259 } } peers { permanent_uuid: "7fb26319cf7544d4b19a81765d7e245a" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 39829 } }
I20260811 06:34:48.235852 26546 leader_election.cc:288] T 3ccb168c9a844cdc9377a3ce7e89b1ef P 7fb26319cf7544d4b19a81765d7e245a [CANDIDATE]: Term 1 election: Requested vote from peers 6c20a80fb808443795647957db66a896 (127.28.239.194:43319), 3f72ffdf95d148b9a12768960ec220b5 (127.28.239.193:34259)
I20260811 06:34:48.236366 26047 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ccb168c9a844cdc9377a3ce7e89b1ef" candidate_uuid: "7fb26319cf7544d4b19a81765d7e245a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f72ffdf95d148b9a12768960ec220b5"
I20260811 06:34:48.236361 26246 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ccb168c9a844cdc9377a3ce7e89b1ef" candidate_uuid: "7fb26319cf7544d4b19a81765d7e245a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c20a80fb808443795647957db66a896"
I20260811 06:34:48.236588 26246 raft_consensus.cc:3037] T 3ccb168c9a844cdc9377a3ce7e89b1ef P 6c20a80fb808443795647957db66a896 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:34:48.236573 26047 raft_consensus.cc:3037] T 3ccb168c9a844cdc9377a3ce7e89b1ef P 3f72ffdf95d148b9a12768960ec220b5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:34:48.238930 26047 raft_consensus.cc:2454] T 3ccb168c9a844cdc9377a3ce7e89b1ef P 3f72ffdf95d148b9a12768960ec220b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fb26319cf7544d4b19a81765d7e245a in term 1.
I20260811 06:34:48.238933 26246 raft_consensus.cc:2454] T 3ccb168c9a844cdc9377a3ce7e89b1ef P 6c20a80fb808443795647957db66a896 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fb26319cf7544d4b19a81765d7e245a in term 1.
I20260811 06:34:48.239390 26339 leader_election.cc:302] T 3ccb168c9a844cdc9377a3ce7e89b1ef P 7fb26319cf7544d4b19a81765d7e245a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f72ffdf95d148b9a12768960ec220b5, 7fb26319cf7544d4b19a81765d7e245a; no voters:
I20260811 06:34:48.239751 26546 raft_consensus.cc:2781] T 3ccb168c9a844cdc9377a3ce7e89b1ef P 7fb26319cf7544d4b19a81765d7e245a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:34:48.240274 26546 raft_consensus.cc:686] T 3ccb168c9a844cdc9377a3ce7e89b1ef P 7fb26319cf7544d4b19a81765d7e245a [term 1 LEADER]: Becoming Leader. State: Replica: 7fb26319cf7544d4b19a81765d7e245a, State: Running, Role: LEADER
I20260811 06:34:48.240876 26546 consensus_queue.cc:237] T 3ccb168c9a844cdc9377a3ce7e89b1ef P 7fb26319cf7544d4b19a81765d7e245a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c20a80fb808443795647957db66a896" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 43319 } } peers { permanent_uuid: "3f72ffdf95d148b9a12768960ec220b5" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 34259 } } peers { permanent_uuid: "7fb26319cf7544d4b19a81765d7e245a" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 39829 } }
I20260811 06:34:48.245713 25838 catalog_manager.cc:5495] T 3ccb168c9a844cdc9377a3ce7e89b1ef P 7fb26319cf7544d4b19a81765d7e245a reported cstate change: term changed from 0 to 1, leader changed from <none> to 7fb26319cf7544d4b19a81765d7e245a (127.28.239.195). New cstate: current_term: 1 leader_uuid: "7fb26319cf7544d4b19a81765d7e245a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c20a80fb808443795647957db66a896" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 43319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f72ffdf95d148b9a12768960ec220b5" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 34259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fb26319cf7544d4b19a81765d7e245a" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 39829 } health_report { overall_health: HEALTHY } } }
I20260811 06:34:48.283514 26536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:48.277839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37724 (local address 127.28.239.195:39829)
0811 06:34:48.278031 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 06:34:48.278035 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:34:48.278052 (+ 17us) server_negotiation.cc:407] Connection header received
0811 06:34:48.278144 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:34:48.278148 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:34:48.278233 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:34:48.278362 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:34:48.278806 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:48.279554 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:48.281035 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:48.281442 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:48.281622 (+ 180us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:48.282311 (+ 689us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:34:48.282898 (+ 587us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:34:48.283119 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0811 06:34:48.283219 (+ 100us) server_negotiation.cc:299] Negotiation successful
0811 06:34:48.283275 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 06:34:48.292526 26047 raft_consensus.cc:1261] T 3ccb168c9a844cdc9377a3ce7e89b1ef P 3f72ffdf95d148b9a12768960ec220b5 [term 1 FOLLOWER]: Refusing update from remote peer 7fb26319cf7544d4b19a81765d7e245a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:34:48.292553 26246 raft_consensus.cc:1261] T 3ccb168c9a844cdc9377a3ce7e89b1ef P 6c20a80fb808443795647957db66a896 [term 1 FOLLOWER]: Refusing update from remote peer 7fb26319cf7544d4b19a81765d7e245a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:34:48.293697 26551 consensus_queue.cc:1035] T 3ccb168c9a844cdc9377a3ce7e89b1ef P 7fb26319cf7544d4b19a81765d7e245a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f72ffdf95d148b9a12768960ec220b5" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 34259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:34:48.294090 26546 consensus_queue.cc:1035] T 3ccb168c9a844cdc9377a3ce7e89b1ef P 7fb26319cf7544d4b19a81765d7e245a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c20a80fb808443795647957db66a896" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 43319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:34:48.322528 25838 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60386:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:34:48.322830 25838 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60386:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:34:48.326632 25838 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8c46112b50dc480fad0a4432e1ee7d99: Sending DeleteTablet for 3 replicas of tablet 3ccb168c9a844cdc9377a3ce7e89b1ef
I20260811 06:34:48.327927 26212 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ccb168c9a844cdc9377a3ce7e89b1ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:34:48 UTC) from {username='kudu', principal='kudu/127.28.239.254@KRBTEST.COM'} at 127.0.0.1:55592
I20260811 06:34:48.327941 26022 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ccb168c9a844cdc9377a3ce7e89b1ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:34:48 UTC) from {username='kudu', principal='kudu/127.28.239.254@KRBTEST.COM'} at 127.0.0.1:55260
I20260811 06:34:48.327941 26407 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ccb168c9a844cdc9377a3ce7e89b1ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:34:48 UTC) from {username='kudu', principal='kudu/127.28.239.254@KRBTEST.COM'} at 127.0.0.1:37720
I20260811 06:34:48.330327 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 25925
I20260811 06:34:48.340900 26573 tablet_replica.cc:330] T 3ccb168c9a844cdc9377a3ce7e89b1ef P 3f72ffdf95d148b9a12768960ec220b5: stopping tablet replica
I20260811 06:34:48.341575 26573 raft_consensus.cc:2229] T 3ccb168c9a844cdc9377a3ce7e89b1ef P 3f72ffdf95d148b9a12768960ec220b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:34:48.342325 26573 raft_consensus.cc:2258] T 3ccb168c9a844cdc9377a3ce7e89b1ef P 3f72ffdf95d148b9a12768960ec220b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:34:48.343403 26571 tablet_replica.cc:330] T 3ccb168c9a844cdc9377a3ce7e89b1ef P 7fb26319cf7544d4b19a81765d7e245a: stopping tablet replica
I20260811 06:34:48.343869 26571 raft_consensus.cc:2229] T 3ccb168c9a844cdc9377a3ce7e89b1ef P 7fb26319cf7544d4b19a81765d7e245a [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:34:48.344682 26571 raft_consensus.cc:2258] T 3ccb168c9a844cdc9377a3ce7e89b1ef P 7fb26319cf7544d4b19a81765d7e245a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:34:48.344776 26573 ts_tablet_manager.cc:1905] T 3ccb168c9a844cdc9377a3ce7e89b1ef P 3f72ffdf95d148b9a12768960ec220b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:34:48.345817 26575 tablet_replica.cc:330] T 3ccb168c9a844cdc9377a3ce7e89b1ef P 6c20a80fb808443795647957db66a896: stopping tablet replica
I20260811 06:34:48.346215 26575 raft_consensus.cc:2229] T 3ccb168c9a844cdc9377a3ce7e89b1ef P 6c20a80fb808443795647957db66a896 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:34:48.346712 26575 raft_consensus.cc:2258] T 3ccb168c9a844cdc9377a3ce7e89b1ef P 6c20a80fb808443795647957db66a896 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:34:48.347200 26571 ts_tablet_manager.cc:1905] T 3ccb168c9a844cdc9377a3ce7e89b1ef P 7fb26319cf7544d4b19a81765d7e245a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:34:48.348793 26575 ts_tablet_manager.cc:1905] T 3ccb168c9a844cdc9377a3ce7e89b1ef P 6c20a80fb808443795647957db66a896: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:34:48.352018 26573 ts_tablet_manager.cc:1918] T 3ccb168c9a844cdc9377a3ce7e89b1ef P 3f72ffdf95d148b9a12768960ec220b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:34:48.352201 26573 log.cc:1196] T 3ccb168c9a844cdc9377a3ce7e89b1ef P 3f72ffdf95d148b9a12768960ec220b5: Deleting WAL directory at /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/minicluster-data/ts-0/wal/wals/3ccb168c9a844cdc9377a3ce7e89b1ef
I20260811 06:34:48.352811 26573 ts_tablet_manager.cc:1939] T 3ccb168c9a844cdc9377a3ce7e89b1ef P 3f72ffdf95d148b9a12768960ec220b5: Deleting consensus metadata
I20260811 06:34:48.354828 25809 catalog_manager.cc:4841] TS 3f72ffdf95d148b9a12768960ec220b5 (127.28.239.193:34259): tablet 3ccb168c9a844cdc9377a3ce7e89b1ef (table test-table [id=eb330727b39648f6982f6fce878c0828]) successfully deleted
I20260811 06:34:48.355650 26575 ts_tablet_manager.cc:1918] T 3ccb168c9a844cdc9377a3ce7e89b1ef P 6c20a80fb808443795647957db66a896: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:34:48.355809 26575 log.cc:1196] T 3ccb168c9a844cdc9377a3ce7e89b1ef P 6c20a80fb808443795647957db66a896: Deleting WAL directory at /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/minicluster-data/ts-1/wal/wals/3ccb168c9a844cdc9377a3ce7e89b1ef
I20260811 06:34:48.356374 26575 ts_tablet_manager.cc:1939] T 3ccb168c9a844cdc9377a3ce7e89b1ef P 6c20a80fb808443795647957db66a896: Deleting consensus metadata
I20260811 06:34:48.358171 25814 catalog_manager.cc:4841] TS 6c20a80fb808443795647957db66a896 (127.28.239.194:43319): tablet 3ccb168c9a844cdc9377a3ce7e89b1ef (table test-table [id=eb330727b39648f6982f6fce878c0828]) successfully deleted
I20260811 06:34:48.358429 26571 ts_tablet_manager.cc:1918] T 3ccb168c9a844cdc9377a3ce7e89b1ef P 7fb26319cf7544d4b19a81765d7e245a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:34:48.358606 26571 log.cc:1196] T 3ccb168c9a844cdc9377a3ce7e89b1ef P 7fb26319cf7544d4b19a81765d7e245a: Deleting WAL directory at /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786429755927508-29631-0/minicluster-data/ts-2/wal/wals/3ccb168c9a844cdc9377a3ce7e89b1ef
I20260811 06:34:48.359256 26571 ts_tablet_manager.cc:1939] T 3ccb168c9a844cdc9377a3ce7e89b1ef P 7fb26319cf7544d4b19a81765d7e245a: Deleting consensus metadata
I20260811 06:34:48.360850 25814 catalog_manager.cc:4841] TS 7fb26319cf7544d4b19a81765d7e245a (127.28.239.195:39829): tablet 3ccb168c9a844cdc9377a3ce7e89b1ef (table test-table [id=eb330727b39648f6982f6fce878c0828]) successfully deleted
Aug 11 06:34:48 dist-test-slave-gkw7 krb5kdc[25773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430085, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.239.193@KRBTEST.COM
I20260811 06:34:48.372556 26530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:48.331817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55274 (local address 127.28.239.193:34259)
0811 06:34:48.332005 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 06:34:48.332012 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:34:48.354794 (+ 22782us) server_negotiation.cc:407] Connection header received
0811 06:34:48.355328 (+ 534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:34:48.355335 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:34:48.355429 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:34:48.355507 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:34:48.357518 (+ 2011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:48.358691 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:48.360693 (+ 2002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:48.361336 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:48.361474 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:48.366016 (+ 4542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:34:48.366033 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:34:48.366052 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:34:48.366096 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:34:48.369644 (+ 3548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:34:48.370348 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:34:48.370357 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:34:48.370362 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:34:48.370499 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:34:48.370894 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:34:48.370899 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:34:48.370902 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:34:48.371280 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:34:48.371606 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0811 06:34:48.371928 (+ 322us) server_negotiation.cc:299] Negotiation successful
0811 06:34:48.372163 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 06:34:48.373349 26105 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:34:48.486615 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 25925
I20260811 06:34:48.526726 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 26123
Aug 11 06:34:48 dist-test-slave-gkw7 krb5kdc[25773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430085, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.239.194@KRBTEST.COM
I20260811 06:34:48.542960 26531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:48.527704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55600 (local address 127.28.239.194:43319)
0811 06:34:48.527908 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 06:34:48.527914 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:34:48.527935 (+ 21us) server_negotiation.cc:407] Connection header received
0811 06:34:48.528125 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:34:48.528131 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:34:48.528212 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:34:48.528331 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:34:48.529694 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:48.530456 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:48.531747 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:48.532140 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:48.532274 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:48.536786 (+ 4512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:34:48.536804 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:34:48.536821 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:34:48.536864 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:34:48.539897 (+ 3033us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:34:48.540676 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:34:48.540683 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:34:48.540686 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:34:48.540817 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:34:48.541225 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:34:48.541230 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:34:48.541235 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:34:48.541703 (+ 468us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:34:48.542046 (+ 343us) server_negotiation.cc:1035] Waiting for connection context
0811 06:34:48.542373 (+ 327us) server_negotiation.cc:299] Negotiation successful
0811 06:34:48.542603 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 06:34:48.543633 26304 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:34:48.637668 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 26123
I20260811 06:34:48.666787 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 26318
Aug 11 06:34:48 dist-test-slave-gkw7 krb5kdc[25773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430085, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.239.195@KRBTEST.COM
I20260811 06:34:48.684412 26536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:48.667666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37730 (local address 127.28.239.195:39829)
0811 06:34:48.667841 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 06:34:48.667847 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:34:48.667931 (+ 84us) server_negotiation.cc:407] Connection header received
0811 06:34:48.668138 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:34:48.668148 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:34:48.668237 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:34:48.668359 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:34:48.669778 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:48.670643 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:48.672052 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:48.672594 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:48.672742 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:48.677406 (+ 4664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:34:48.677420 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:34:48.677438 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:34:48.677479 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:34:48.680810 (+ 3331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:34:48.681771 (+ 961us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:34:48.681795 (+ 24us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:34:48.681802 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:34:48.682015 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:34:48.682610 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:34:48.682615 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:34:48.682618 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:34:48.683104 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:34:48.683564 (+ 460us) server_negotiation.cc:1035] Waiting for connection context
0811 06:34:48.683756 (+ 192us) server_negotiation.cc:299] Negotiation successful
0811 06:34:48.684005 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 06:34:48.685204 26501 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:34:48.814934 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 26318
I20260811 06:34:48.844610 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 25788
I20260811 06:34:48.845878 25881 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:34:48.957243 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 25788
2026-08-11T06:34:48Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (5002 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-11T06:34:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:34:49Z Disabled control of system clock
I20260811 06:34:49.062959 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429755927508-29631-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429755927508-29631-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429755927508-29631-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:44445
--webserver_interface=127.28.239.254
--webserver_port=0
--builtin_ntp_servers=127.28.239.212:44163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.239.254:44445
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 06:34:49.422760 26591 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:34:49.423120 26591 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:34:49.423215 26591 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:34:49.430773 26591 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:34:49.430907 26591 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:34:49.430951 26591 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:34:49.431020 26591 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:34:49.431080 26591 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:34:49.440835 26591 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:44163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429755927508-29631-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429755927508-29631-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.239.254:44445
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:44445
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.239.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429755927508-29631-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:34:49.442315 26591 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:34:49.444232 26591 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:34:49.457361 26600 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:34:49.457382 26602 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:34:49.457340 26604 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:34:49.458190 26591 server_base.cc:1034] running on GCE node
I20260811 06:34:49.459084 26591 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:34:49.460507 26591 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:34:49.461750 26591 hybrid_clock.cc:648] HybridClock initialized: now 1786430089461678 us; error 66 us; skew 500 ppm
I20260811 06:34:49.462128 26591 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:34:49.464547 26591 webserver.cc:460] Webserver started at http://127.28.239.254:41559/ using document root <none> and password file <none>
I20260811 06:34:49.465310 26591 fs_manager.cc:359] Metadata directory not provided
I20260811 06:34:49.465445 26591 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:34:49.465812 26591 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:34:49.468946 26591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429755927508-29631-0/minicluster-data/master-0/data/instance:
uuid: "8241ec55d42945e6ad5b8af2d3b68d78"
format_stamp: "Formatted at 2026-08-11 06:34:49 on dist-test-slave-gkw7"
server_key: "9b1396821729949eb0fccb3718b0c1bf"
server_key_iv: "3550c8efa9b3c27356f52cb069e1c7ae"
server_key_version: "encryptionkey@0"
I20260811 06:34:49.469890 26591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429755927508-29631-0/minicluster-data/master-0/wal/instance:
uuid: "8241ec55d42945e6ad5b8af2d3b68d78"
format_stamp: "Formatted at 2026-08-11 06:34:49 on dist-test-slave-gkw7"
server_key: "9b1396821729949eb0fccb3718b0c1bf"
server_key_iv: "3550c8efa9b3c27356f52cb069e1c7ae"
server_key_version: "encryptionkey@0"
I20260811 06:34:49.475176 26591 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 06:34:49.478758 26610 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:34:49.480161 26591 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260811 06:34:49.480360 26591 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429755927508-29631-0/minicluster-data/master-0/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429755927508-29631-0/minicluster-data/master-0/wal
uuid: "8241ec55d42945e6ad5b8af2d3b68d78"
format_stamp: "Formatted at 2026-08-11 06:34:49 on dist-test-slave-gkw7"
server_key: "9b1396821729949eb0fccb3718b0c1bf"
server_key_iv: "3550c8efa9b3c27356f52cb069e1c7ae"
server_key_version: "encryptionkey@0"
I20260811 06:34:49.480600 26591 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429755927508-29631-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429755927508-29631-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429755927508-29631-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:34:49.498051 26591 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:34:49.498980 26591 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:34:49.499306 26591 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:34:49.518925 26591 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.254:44445
I20260811 06:34:49.518965 26686 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.254:44445 every 8 connection(s)
I20260811 06:34:49.520820 26591 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
I20260811 06:34:49.525383 26687 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:34:49.526360 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 26591
I20260811 06:34:49.526551 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429755927508-29631-0/minicluster-data/master-0/wal/instance
I20260811 06:34:49.527051 29631 external_mini_cluster.cc:1428] Setting key b139bca83d03beb49ad6e11d329aeb95
I20260811 06:34:49.538471 26687 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8241ec55d42945e6ad5b8af2d3b68d78: Bootstrap starting.
I20260811 06:34:49.540887 26691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:49.529142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56034 (local address 127.28.239.254:44445)
0811 06:34:49.530313 (+ 1171us) server_negotiation.cc:206] Beginning negotiation
0811 06:34:49.530324 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 06:34:49.530367 (+ 43us) server_negotiation.cc:407] Connection header received
0811 06:34:49.532022 (+ 1655us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:34:49.532046 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:34:49.532529 (+ 483us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:34:49.532949 (+ 420us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:34:49.533600 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:49.534886 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:49.536285 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:49.536989 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:49.537194 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:49.538541 (+ 1347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:34:49.538549 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:34:49.538563 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:34:49.539155 (+ 592us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:34:49.539236 (+ 81us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:34:49.539258 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:34:49.539401 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 06:34:49.539628 (+ 227us) server_negotiation.cc:299] Negotiation successful
0811 06:34:49.539958 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":715,"thread_start_us":480,"threads_started":1}
I20260811 06:34:49.541949 26687 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8241ec55d42945e6ad5b8af2d3b68d78: Neither blocks nor log segments found. Creating new log.
I20260811 06:34:49.543345 26687 log.cc:824] T 00000000000000000000000000000000 P 8241ec55d42945e6ad5b8af2d3b68d78: Log is configured to *not* fsync() on all Append() calls
I20260811 06:34:49.546684 26687 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8241ec55d42945e6ad5b8af2d3b68d78: No bootstrap required, opened a new log
I20260811 06:34:49.554913 26687 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8241ec55d42945e6ad5b8af2d3b68d78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8241ec55d42945e6ad5b8af2d3b68d78" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 44445 } }
I20260811 06:34:49.555298 26687 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8241ec55d42945e6ad5b8af2d3b68d78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:34:49.555378 26687 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8241ec55d42945e6ad5b8af2d3b68d78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8241ec55d42945e6ad5b8af2d3b68d78, State: Initialized, Role: FOLLOWER
I20260811 06:34:49.555883 26687 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8241ec55d42945e6ad5b8af2d3b68d78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8241ec55d42945e6ad5b8af2d3b68d78" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 44445 } }
I20260811 06:34:49.556083 26687 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8241ec55d42945e6ad5b8af2d3b68d78 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:34:49.556191 26687 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8241ec55d42945e6ad5b8af2d3b68d78 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:34:49.556386 26687 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8241ec55d42945e6ad5b8af2d3b68d78 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:34:49.558733 26687 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8241ec55d42945e6ad5b8af2d3b68d78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8241ec55d42945e6ad5b8af2d3b68d78" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 44445 } }
I20260811 06:34:49.559265 26687 leader_election.cc:302] T 00000000000000000000000000000000 P 8241ec55d42945e6ad5b8af2d3b68d78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8241ec55d42945e6ad5b8af2d3b68d78; no voters:
I20260811 06:34:49.560014 26687 leader_election.cc:288] T 00000000000000000000000000000000 P 8241ec55d42945e6ad5b8af2d3b68d78 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:34:49.560137 26694 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8241ec55d42945e6ad5b8af2d3b68d78 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:34:49.560876 26694 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8241ec55d42945e6ad5b8af2d3b68d78 [term 1 LEADER]: Becoming Leader. State: Replica: 8241ec55d42945e6ad5b8af2d3b68d78, State: Running, Role: LEADER
I20260811 06:34:49.561481 26694 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8241ec55d42945e6ad5b8af2d3b68d78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8241ec55d42945e6ad5b8af2d3b68d78" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 44445 } }
I20260811 06:34:49.562525 26687 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8241ec55d42945e6ad5b8af2d3b68d78 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:34:49.564612 26696 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8241ec55d42945e6ad5b8af2d3b68d78 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8241ec55d42945e6ad5b8af2d3b68d78. Latest consensus state: current_term: 1 leader_uuid: "8241ec55d42945e6ad5b8af2d3b68d78" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8241ec55d42945e6ad5b8af2d3b68d78" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 44445 } } }
I20260811 06:34:49.564714 26695 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8241ec55d42945e6ad5b8af2d3b68d78 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8241ec55d42945e6ad5b8af2d3b68d78" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8241ec55d42945e6ad5b8af2d3b68d78" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 44445 } } }
I20260811 06:34:49.565117 26695 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8241ec55d42945e6ad5b8af2d3b68d78 [sys.catalog]: This master's current role is: LEADER
I20260811 06:34:49.565114 26696 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8241ec55d42945e6ad5b8af2d3b68d78 [sys.catalog]: This master's current role is: LEADER
I20260811 06:34:49.567112 26699 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:34:49.573699 26699 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:34:49.584133 26699 catalog_manager.cc:1347] Generated new cluster ID: f2787209e8ff43e9afb84857e16af6db
I20260811 06:34:49.584307 26699 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:34:49.591908 26699 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:34:49.593196 26699 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:34:49.605397 26699 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8241ec55d42945e6ad5b8af2d3b68d78: Generated new TSK 0
I20260811 06:34:49.606350 26699 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:34:49.625089 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429755927508-29631-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429755927508-29631-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429755927508-29631-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.193:0
--local_ip_for_outbound_sockets=127.28.239.193
--webserver_interface=127.28.239.193
--webserver_port=0
--tserver_master_addrs=127.28.239.254:44445
--builtin_ntp_servers=127.28.239.212:44163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 06:34:49.977653 26717 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:34:49.977995 26717 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:34:49.978178 26717 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:34:49.985705 26717 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:34:49.985854 26717 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:34:49.986037 26717 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.193
I20260811 06:34:49.995383 26717 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:44163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429755927508-29631-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429755927508-29631-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.239.193
--webserver_port=0
--tserver_master_addrs=127.28.239.254:44445
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.193
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429755927508-29631-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:34:49.996789 26717 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:34:49.998694 26717 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:34:50.012465 26725 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:34:50.012516 26726 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:34:50.012704 26728 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:34:50.013837 26717 server_base.cc:1034] running on GCE node
I20260811 06:34:50.014503 26717 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:34:50.015506 26717 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:34:50.016750 26717 hybrid_clock.cc:648] HybridClock initialized: now 1786430090016698 us; error 65 us; skew 500 ppm
I20260811 06:34:50.017081 26717 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:34:50.019435 26717 webserver.cc:460] Webserver started at http://127.28.239.193:42403/ using document root <none> and password file <none>
I20260811 06:34:50.020153 26717 fs_manager.cc:359] Metadata directory not provided
I20260811 06:34:50.020304 26717 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:34:50.020628 26717 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:34:50.023623 26717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429755927508-29631-0/minicluster-data/ts-0/data/instance:
uuid: "7b8f858dd216415bbdfab182e00a7eb1"
format_stamp: "Formatted at 2026-08-11 06:34:50 on dist-test-slave-gkw7"
server_key: "e5374387e061f596fb337b4c76eb7262"
server_key_iv: "6c21f99a9953b104856163deef1bc141"
server_key_version: "encryptionkey@0"
I20260811 06:34:50.024456 26717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429755927508-29631-0/minicluster-data/ts-0/wal/instance:
uuid: "7b8f858dd216415bbdfab182e00a7eb1"
format_stamp: "Formatted at 2026-08-11 06:34:50 on dist-test-slave-gkw7"
server_key: "e5374387e061f596fb337b4c76eb7262"
server_key_iv: "6c21f99a9953b104856163deef1bc141"
server_key_version: "encryptionkey@0"
I20260811 06:34:50.029784 26717 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260811 06:34:50.033203 26735 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:34:50.034641 26717 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:34:50.034825 26717 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429755927508-29631-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429755927508-29631-0/minicluster-data/ts-0/wal
uuid: "7b8f858dd216415bbdfab182e00a7eb1"
format_stamp: "Formatted at 2026-08-11 06:34:50 on dist-test-slave-gkw7"
server_key: "e5374387e061f596fb337b4c76eb7262"
server_key_iv: "6c21f99a9953b104856163deef1bc141"
server_key_version: "encryptionkey@0"
I20260811 06:34:50.035050 26717 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429755927508-29631-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429755927508-29631-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429755927508-29631-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:34:50.055269 26717 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:34:50.056133 26717 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:34:50.056466 26717 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:34:50.057499 26717 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:34:50.059067 26717 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:34:50.059208 26717 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:34:50.059340 26717 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:34:50.059429 26717 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:34:50.094771 26717 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.193:38067
I20260811 06:34:50.094864 26907 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.193:38067 every 8 connection(s)
I20260811 06:34:50.096247 26717 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
I20260811 06:34:50.099381 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 26717
I20260811 06:34:50.099571 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429755927508-29631-0/minicluster-data/ts-0/wal/instance
I20260811 06:34:50.100029 29631 external_mini_cluster.cc:1428] Setting key cf1d69adca4bdfbcd11951665cc15848
I20260811 06:34:50.102873 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429755927508-29631-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429755927508-29631-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429755927508-29631-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.194:0
--local_ip_for_outbound_sockets=127.28.239.194
--webserver_interface=127.28.239.194
--webserver_port=0
--tserver_master_addrs=127.28.239.254:44445
--builtin_ntp_servers=127.28.239.212:44163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 06:34:50.111404 26912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:50.099049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.193:36373 (local address 127.28.239.254:44445)
0811 06:34:50.099591 (+ 542us) server_negotiation.cc:206] Beginning negotiation
0811 06:34:50.099600 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:34:50.101618 (+ 2018us) server_negotiation.cc:407] Connection header received
0811 06:34:50.103003 (+ 1385us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:34:50.103008 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:34:50.103099 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:34:50.103212 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:34:50.104777 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:50.105759 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:50.108232 (+ 2473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:50.108662 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:50.108762 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:50.110472 (+ 1710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:34:50.110477 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:34:50.110480 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:34:50.110499 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:34:50.110539 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:34:50.110550 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:34:50.110626 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 06:34:50.110910 (+ 284us) server_negotiation.cc:299] Negotiation successful
0811 06:34:50.110989 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":252,"threads_started":1}
I20260811 06:34:50.112907 26913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:50.099802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:44445 (local address 127.28.239.193:36373)
0811 06:34:50.100892 (+ 1090us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:50.100947 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 06:34:50.102709 (+ 1762us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:34:50.103485 (+ 776us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:34:50.103499 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:34:50.103622 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:34:50.104541 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:50.104572 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:50.105918 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:50.105926 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:50.108079 (+ 2153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:50.108098 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:50.108898 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:50.108908 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:50.109247 (+ 339us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:50.110071 (+ 824us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 06:34:50.110077 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:34:50.110135 (+ 58us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 06:34:50.110257 (+ 122us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:34:50.110664 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:34:50.110680 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:34:50.110692 (+ 12us) client_negotiation.cc:769] Sending connection context
0811 06:34:50.110885 (+ 193us) client_negotiation.cc:240] Negotiation successful
0811 06:34:50.111894 (+ 1009us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":356,"threads_started":1}
I20260811 06:34:50.114563 26908 heartbeater.cc:347] Connected to a master server at 127.28.239.254:44445
I20260811 06:34:50.114902 26908 heartbeater.cc:464] Registering TS with master...
I20260811 06:34:50.115707 26908 heartbeater.cc:511] Master 127.28.239.254:44445 requested a full tablet report, sending...
I20260811 06:34:50.118108 26633 ts_manager.cc:194] Registered new tserver with Master: 7b8f858dd216415bbdfab182e00a7eb1 (127.28.239.193:38067)
I20260811 06:34:50.119625 26633 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.28.239.193:36373
W20260811 06:34:50.449913 26914 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:34:50.450263 26914 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:34:50.450435 26914 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:34:50.458074 26914 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:34:50.458218 26914 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:34:50.458398 26914 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.194
I20260811 06:34:50.467800 26914 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:44163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429755927508-29631-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429755927508-29631-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.239.194
--webserver_port=0
--tserver_master_addrs=127.28.239.254:44445
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.194
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429755927508-29631-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:34:50.469256 26914 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:34:50.471298 26914 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:34:50.485348 26923 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:34:50.485276 26926 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:34:50.485662 26921 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:34:50.486357 26914 server_base.cc:1034] running on GCE node
I20260811 06:34:50.486987 26914 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:34:50.488062 26914 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:34:50.489282 26914 hybrid_clock.cc:648] HybridClock initialized: now 1786430090489222 us; error 51 us; skew 500 ppm
I20260811 06:34:50.489692 26914 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:34:50.492060 26914 webserver.cc:460] Webserver started at http://127.28.239.194:41147/ using document root <none> and password file <none>
I20260811 06:34:50.492810 26914 fs_manager.cc:359] Metadata directory not provided
I20260811 06:34:50.492937 26914 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:34:50.493292 26914 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:34:50.496354 26914 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429755927508-29631-0/minicluster-data/ts-1/data/instance:
uuid: "5e5721de04ec4e0d9a2f079e38ee196e"
format_stamp: "Formatted at 2026-08-11 06:34:50 on dist-test-slave-gkw7"
server_key: "3cd5f1e61a75b206e253383110d84484"
server_key_iv: "d4c5123dbbe2eb1408f0150e7d092207"
server_key_version: "encryptionkey@0"
I20260811 06:34:50.497162 26914 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429755927508-29631-0/minicluster-data/ts-1/wal/instance:
uuid: "5e5721de04ec4e0d9a2f079e38ee196e"
format_stamp: "Formatted at 2026-08-11 06:34:50 on dist-test-slave-gkw7"
server_key: "3cd5f1e61a75b206e253383110d84484"
server_key_iv: "d4c5123dbbe2eb1408f0150e7d092207"
server_key_version: "encryptionkey@0"
I20260811 06:34:50.502583 26914 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 06:34:50.506160 26933 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:34:50.507632 26914 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 06:34:50.507818 26914 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429755927508-29631-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429755927508-29631-0/minicluster-data/ts-1/wal
uuid: "5e5721de04ec4e0d9a2f079e38ee196e"
format_stamp: "Formatted at 2026-08-11 06:34:50 on dist-test-slave-gkw7"
server_key: "3cd5f1e61a75b206e253383110d84484"
server_key_iv: "d4c5123dbbe2eb1408f0150e7d092207"
server_key_version: "encryptionkey@0"
I20260811 06:34:50.508042 26914 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429755927508-29631-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429755927508-29631-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429755927508-29631-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:34:50.541770 26914 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:34:50.542717 26914 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:34:50.543054 26914 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:34:50.544126 26914 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:34:50.546171 26914 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:34:50.546306 26914 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:34:50.546440 26914 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:34:50.546530 26914 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:34:50.581491 26914 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.194:38033
I20260811 06:34:50.581626 27098 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.194:38033 every 8 connection(s)
I20260811 06:34:50.582959 26914 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
I20260811 06:34:50.589377 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 26914
I20260811 06:34:50.589602 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429755927508-29631-0/minicluster-data/ts-1/wal/instance
I20260811 06:34:50.590104 29631 external_mini_cluster.cc:1428] Setting key 16ffdbcc305f982cc879121b3af26eae
I20260811 06:34:50.592939 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429755927508-29631-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429755927508-29631-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429755927508-29631-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.195:0
--local_ip_for_outbound_sockets=127.28.239.195
--webserver_interface=127.28.239.195
--webserver_port=0
--tserver_master_addrs=127.28.239.254:44445
--builtin_ntp_servers=127.28.239.212:44163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 06:34:50.598832 26912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:50.585973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.194:36501 (local address 127.28.239.254:44445)
0811 06:34:50.586182 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 06:34:50.586187 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:34:50.587690 (+ 1503us) server_negotiation.cc:407] Connection header received
0811 06:34:50.589553 (+ 1863us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:34:50.589560 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:34:50.589696 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:34:50.589806 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:34:50.591140 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:50.591982 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:50.595803 (+ 3821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:50.596246 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:50.596350 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:50.598073 (+ 1723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:34:50.598081 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:34:50.598085 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:34:50.598123 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:34:50.598156 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:34:50.598169 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:34:50.598244 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 06:34:50.598520 (+ 276us) server_negotiation.cc:299] Negotiation successful
0811 06:34:50.598606 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 06:34:50.600448 27101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:50.586562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:44445 (local address 127.28.239.194:36501)
0811 06:34:50.587515 (+ 953us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:50.587563 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 06:34:50.589252 (+ 1689us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:34:50.590053 (+ 801us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:34:50.590062 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:34:50.590152 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:34:50.590960 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:50.590981 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:50.593287 (+ 2306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:50.593296 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:50.595628 (+ 2332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:50.595647 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:50.596490 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:50.596501 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:50.596840 (+ 339us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:50.597662 (+ 822us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 06:34:50.597668 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:34:50.597727 (+ 59us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 06:34:50.597836 (+ 109us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:34:50.598283 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:34:50.598298 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:34:50.598309 (+ 11us) client_negotiation.cc:769] Sending connection context
0811 06:34:50.598507 (+ 198us) client_negotiation.cc:240] Negotiation successful
0811 06:34:50.598960 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":302,"threads_started":1}
I20260811 06:34:50.602104 27099 heartbeater.cc:347] Connected to a master server at 127.28.239.254:44445
I20260811 06:34:50.602419 27099 heartbeater.cc:464] Registering TS with master...
I20260811 06:34:50.603211 27099 heartbeater.cc:511] Master 127.28.239.254:44445 requested a full tablet report, sending...
I20260811 06:34:50.605010 26633 ts_manager.cc:194] Registered new tserver with Master: 5e5721de04ec4e0d9a2f079e38ee196e (127.28.239.194:38033)
I20260811 06:34:50.606328 26633 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.28.239.194:36501
W20260811 06:34:50.966421 27102 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:34:50.966737 27102 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:34:50.966939 27102 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:34:50.974603 27102 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:34:50.974742 27102 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:34:50.974928 27102 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.195
I20260811 06:34:50.984831 27102 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:44163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429755927508-29631-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429755927508-29631-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.239.195
--webserver_port=0
--tserver_master_addrs=127.28.239.254:44445
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.195
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429755927508-29631-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:34:50.986366 27102 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:34:50.988336 27102 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:34:51.002920 27112 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:34:51.002867 27115 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:34:51.002959 27110 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:34:51.003957 27102 server_base.cc:1034] running on GCE node
I20260811 06:34:51.004739 27102 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:34:51.005910 27102 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:34:51.007126 27102 hybrid_clock.cc:648] HybridClock initialized: now 1786430091007069 us; error 47 us; skew 500 ppm
I20260811 06:34:51.007495 27102 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:34:51.010095 27102 webserver.cc:460] Webserver started at http://127.28.239.195:41619/ using document root <none> and password file <none>
I20260811 06:34:51.010879 27102 fs_manager.cc:359] Metadata directory not provided
I20260811 06:34:51.010978 27102 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:34:51.011441 27102 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:34:51.014847 27102 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429755927508-29631-0/minicluster-data/ts-2/data/instance:
uuid: "ca76b697a3ea40c1bb128ac5e5744598"
format_stamp: "Formatted at 2026-08-11 06:34:51 on dist-test-slave-gkw7"
server_key: "52cfd8661eebc5ad9534bc4cedb484a5"
server_key_iv: "275e7f1553e5aa9babb639753e5022eb"
server_key_version: "encryptionkey@0"
I20260811 06:34:51.015702 27102 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429755927508-29631-0/minicluster-data/ts-2/wal/instance:
uuid: "ca76b697a3ea40c1bb128ac5e5744598"
format_stamp: "Formatted at 2026-08-11 06:34:51 on dist-test-slave-gkw7"
server_key: "52cfd8661eebc5ad9534bc4cedb484a5"
server_key_iv: "275e7f1553e5aa9babb639753e5022eb"
server_key_version: "encryptionkey@0"
I20260811 06:34:51.021306 27102 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260811 06:34:51.024997 27121 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:34:51.026466 27102 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:34:51.026652 27102 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429755927508-29631-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429755927508-29631-0/minicluster-data/ts-2/wal
uuid: "ca76b697a3ea40c1bb128ac5e5744598"
format_stamp: "Formatted at 2026-08-11 06:34:51 on dist-test-slave-gkw7"
server_key: "52cfd8661eebc5ad9534bc4cedb484a5"
server_key_iv: "275e7f1553e5aa9babb639753e5022eb"
server_key_version: "encryptionkey@0"
I20260811 06:34:51.026907 27102 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429755927508-29631-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429755927508-29631-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429755927508-29631-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:34:51.052727 27102 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:34:51.053710 27102 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:34:51.054071 27102 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:34:51.055339 27102 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:34:51.056957 27102 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:34:51.057087 27102 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:34:51.057222 27102 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:34:51.057315 27102 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:34:51.092949 27102 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.195:36355
I20260811 06:34:51.093052 27296 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.195:36355 every 8 connection(s)
I20260811 06:34:51.094630 27102 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
I20260811 06:34:51.100430 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 27102
I20260811 06:34:51.100625 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429755927508-29631-0/minicluster-data/ts-2/wal/instance
I20260811 06:34:51.101200 29631 external_mini_cluster.cc:1428] Setting key 78e5f24c34c1ef87bf1e9666c79eae8f
I20260811 06:34:51.109243 26912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:51.097615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.195:49333 (local address 127.28.239.254:44445)
0811 06:34:51.097830 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 06:34:51.097835 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:34:51.099499 (+ 1664us) server_negotiation.cc:407] Connection header received
0811 06:34:51.102299 (+ 2800us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:34:51.102305 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:34:51.102402 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:34:51.102518 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:34:51.103870 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:51.104789 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:51.106450 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:51.106925 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:51.107040 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:51.108464 (+ 1424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:34:51.108469 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:34:51.108473 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:34:51.108492 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:34:51.108534 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:34:51.108547 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:34:51.108631 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 06:34:51.108845 (+ 214us) server_negotiation.cc:299] Negotiation successful
0811 06:34:51.108928 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260811 06:34:51.110337 27299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:51.098290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:44445 (local address 127.28.239.195:49333)
0811 06:34:51.099293 (+ 1003us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:51.099367 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0811 06:34:51.101905 (+ 2538us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:34:51.102832 (+ 927us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:34:51.102843 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:34:51.102973 (+ 130us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:34:51.103704 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:51.103721 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:51.104932 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:51.104936 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:51.106293 (+ 1357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:51.106305 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:51.107140 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:51.107147 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:51.107356 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:51.108099 (+ 743us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 06:34:51.108102 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:34:51.108177 (+ 75us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 06:34:51.108266 (+ 89us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:34:51.108649 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:34:51.108658 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:34:51.108664 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 06:34:51.108810 (+ 146us) client_negotiation.cc:240] Negotiation successful
0811 06:34:51.109113 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":354,"threads_started":1}
I20260811 06:34:51.112078 27297 heartbeater.cc:347] Connected to a master server at 127.28.239.254:44445
I20260811 06:34:51.112412 27297 heartbeater.cc:464] Registering TS with master...
I20260811 06:34:51.113269 27297 heartbeater.cc:511] Master 127.28.239.254:44445 requested a full tablet report, sending...
I20260811 06:34:51.115025 26633 ts_manager.cc:194] Registered new tserver with Master: ca76b697a3ea40c1bb128ac5e5744598 (127.28.239.195:36355)
I20260811 06:34:51.116230 26633 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.28.239.195:49333
I20260811 06:34:51.116991 29631 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 06:34:51.123839 26908 heartbeater.cc:503] Master 127.28.239.254:44445 was elected leader, sending a full tablet report...
I20260811 06:34:51.128427 26912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:51.121220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56040 (local address 127.28.239.254:44445)
0811 06:34:51.121444 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0811 06:34:51.121449 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:34:51.121993 (+ 544us) server_negotiation.cc:407] Connection header received
0811 06:34:51.122124 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:34:51.122128 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:34:51.122205 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:34:51.122308 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:34:51.122812 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:51.123522 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:51.125799 (+ 2277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:51.126246 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:51.126405 (+ 159us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:51.127625 (+ 1220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:34:51.127630 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:34:51.127634 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:34:51.127653 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:34:51.127686 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:34:51.127697 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:34:51.127871 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0811 06:34:51.128076 (+ 205us) server_negotiation.cc:299] Negotiation successful
0811 06:34:51.128138 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260811 06:34:51.133021 26633 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56040:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:34:51.140530 26633 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:34:51.163579 27321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:51.153383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48054 (local address 127.28.239.194:38033)
0811 06:34:51.154111 (+ 728us) server_negotiation.cc:206] Beginning negotiation
0811 06:34:51.154123 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 06:34:51.155023 (+ 900us) server_negotiation.cc:407] Connection header received
0811 06:34:51.155546 (+ 523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:34:51.155559 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:34:51.155886 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:34:51.156063 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:34:51.156960 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:51.158404 (+ 1444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:51.160323 (+ 1919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:51.161915 (+ 1592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:51.162041 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:51.162253 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0811 06:34:51.162994 (+ 741us) server_negotiation.cc:299] Negotiation successful
0811 06:34:51.163178 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":545,"thread_start_us":655,"threads_started":1}
I20260811 06:34:51.164569 27322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:51.153204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.194:38033 (local address 127.0.0.1:48054)
0811 06:34:51.154841 (+ 1637us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:51.154890 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 06:34:51.155241 (+ 351us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:34:51.156337 (+ 1096us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:34:51.156342 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:34:51.156366 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:34:51.156725 (+ 359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:51.156740 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:51.158556 (+ 1816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:51.158559 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:51.160091 (+ 1532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:51.160105 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:51.162475 (+ 2370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:51.162485 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:51.162678 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:51.162715 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 06:34:51.163902 (+ 1187us) client_negotiation.cc:240] Negotiation successful
0811 06:34:51.164157 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1283,"thread_start_us":2808,"threads_started":1}
I20260811 06:34:51.165557 27323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:51.153288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.195:36355 (local address 127.0.0.1:35144)
0811 06:34:51.155391 (+ 2103us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:51.155412 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 06:34:51.155536 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:34:51.156776 (+ 1240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:34:51.156786 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:34:51.156814 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:34:51.157220 (+ 406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:51.157234 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:51.160253 (+ 3019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:51.160261 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:51.162905 (+ 2644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:51.162920 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:51.164577 (+ 1657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:51.164584 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:51.164957 (+ 373us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:51.164982 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 06:34:51.165092 (+ 110us) client_negotiation.cc:240] Negotiation successful
0811 06:34:51.165239 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1942,"thread_start_us":1013,"threads_started":1}
I20260811 06:34:51.165756 27325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:51.153887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35144 (local address 127.28.239.195:36355)
0811 06:34:51.156148 (+ 2261us) server_negotiation.cc:206] Beginning negotiation
0811 06:34:51.156157 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:34:51.156193 (+ 36us) server_negotiation.cc:407] Connection header received
0811 06:34:51.156311 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:34:51.156320 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:34:51.156559 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:34:51.156771 (+ 212us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:34:51.158237 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:51.160097 (+ 1860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:51.163116 (+ 3019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:51.164306 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:51.164517 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:51.164787 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0811 06:34:51.165153 (+ 366us) server_negotiation.cc:299] Negotiation successful
0811 06:34:51.165314 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2083,"thread_start_us":1915,"threads_started":1}
I20260811 06:34:51.169119 27005 tablet_service.cc:1467] Processing CreateTablet for tablet ded9e07167a2429881175e61ae72f52e (DEFAULT_TABLE table=test-table [id=ac8479e9a04143c587f6e8bfef4847b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:34:51.169422 27203 tablet_service.cc:1467] Processing CreateTablet for tablet ded9e07167a2429881175e61ae72f52e (DEFAULT_TABLE table=test-table [id=ac8479e9a04143c587f6e8bfef4847b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:34:51.170370 27326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:51.155161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.193:38067 (local address 127.0.0.1:42036)
0811 06:34:51.162684 (+ 7523us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:51.162703 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 06:34:51.162842 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:34:51.163799 (+ 957us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:34:51.163805 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:34:51.163833 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:34:51.164247 (+ 414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:51.164259 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:51.166762 (+ 2503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:51.166768 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:51.168245 (+ 1477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:51.168259 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:51.169625 (+ 1366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:51.169632 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:51.169801 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:51.169838 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 06:34:51.169929 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 06:34:51.170055 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7382,"thread_start_us":7154,"threads_started":1}
I20260811 06:34:51.170502 27327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:51.155815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42036 (local address 127.28.239.193:38067)
0811 06:34:51.160746 (+ 4931us) server_negotiation.cc:206] Beginning negotiation
0811 06:34:51.160760 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 06:34:51.162838 (+ 2078us) server_negotiation.cc:407] Connection header received
0811 06:34:51.163027 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:34:51.163036 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:34:51.163264 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:34:51.163504 (+ 240us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:34:51.164520 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:51.166576 (+ 2056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:51.168452 (+ 1876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:51.169393 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:51.169634 (+ 241us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:51.169897 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0811 06:34:51.169977 (+ 80us) server_negotiation.cc:299] Negotiation successful
0811 06:34:51.170130 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4730,"thread_start_us":4577,"threads_started":1}
I20260811 06:34:51.170991 27203 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ded9e07167a2429881175e61ae72f52e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:34:51.171263 27005 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ded9e07167a2429881175e61ae72f52e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:34:51.173799 26815 tablet_service.cc:1467] Processing CreateTablet for tablet ded9e07167a2429881175e61ae72f52e (DEFAULT_TABLE table=test-table [id=ac8479e9a04143c587f6e8bfef4847b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:34:51.175032 26815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ded9e07167a2429881175e61ae72f52e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:34:51.186851 27330 tablet_bootstrap.cc:492] T ded9e07167a2429881175e61ae72f52e P ca76b697a3ea40c1bb128ac5e5744598: Bootstrap starting.
I20260811 06:34:51.188422 27332 tablet_bootstrap.cc:492] T ded9e07167a2429881175e61ae72f52e P 5e5721de04ec4e0d9a2f079e38ee196e: Bootstrap starting.
I20260811 06:34:51.188686 27333 tablet_bootstrap.cc:492] T ded9e07167a2429881175e61ae72f52e P 7b8f858dd216415bbdfab182e00a7eb1: Bootstrap starting.
I20260811 06:34:51.190791 27330 tablet_bootstrap.cc:654] T ded9e07167a2429881175e61ae72f52e P ca76b697a3ea40c1bb128ac5e5744598: Neither blocks nor log segments found. Creating new log.
I20260811 06:34:51.191847 27330 log.cc:824] T ded9e07167a2429881175e61ae72f52e P ca76b697a3ea40c1bb128ac5e5744598: Log is configured to *not* fsync() on all Append() calls
I20260811 06:34:51.192162 27333 tablet_bootstrap.cc:654] T ded9e07167a2429881175e61ae72f52e P 7b8f858dd216415bbdfab182e00a7eb1: Neither blocks nor log segments found. Creating new log.
I20260811 06:34:51.192162 27332 tablet_bootstrap.cc:654] T ded9e07167a2429881175e61ae72f52e P 5e5721de04ec4e0d9a2f079e38ee196e: Neither blocks nor log segments found. Creating new log.
I20260811 06:34:51.193432 27333 log.cc:824] T ded9e07167a2429881175e61ae72f52e P 7b8f858dd216415bbdfab182e00a7eb1: Log is configured to *not* fsync() on all Append() calls
I20260811 06:34:51.193432 27332 log.cc:824] T ded9e07167a2429881175e61ae72f52e P 5e5721de04ec4e0d9a2f079e38ee196e: Log is configured to *not* fsync() on all Append() calls
I20260811 06:34:51.195363 27330 tablet_bootstrap.cc:492] T ded9e07167a2429881175e61ae72f52e P ca76b697a3ea40c1bb128ac5e5744598: No bootstrap required, opened a new log
I20260811 06:34:51.195675 27330 ts_tablet_manager.cc:1397] T ded9e07167a2429881175e61ae72f52e P ca76b697a3ea40c1bb128ac5e5744598: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.004s
I20260811 06:34:51.198168 27333 tablet_bootstrap.cc:492] T ded9e07167a2429881175e61ae72f52e P 7b8f858dd216415bbdfab182e00a7eb1: No bootstrap required, opened a new log
I20260811 06:34:51.198467 27333 ts_tablet_manager.cc:1397] T ded9e07167a2429881175e61ae72f52e P 7b8f858dd216415bbdfab182e00a7eb1: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.004s
I20260811 06:34:51.198688 27332 tablet_bootstrap.cc:492] T ded9e07167a2429881175e61ae72f52e P 5e5721de04ec4e0d9a2f079e38ee196e: No bootstrap required, opened a new log
I20260811 06:34:51.198959 27332 ts_tablet_manager.cc:1397] T ded9e07167a2429881175e61ae72f52e P 5e5721de04ec4e0d9a2f079e38ee196e: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.004s
I20260811 06:34:51.203352 27330 raft_consensus.cc:348] T ded9e07167a2429881175e61ae72f52e P ca76b697a3ea40c1bb128ac5e5744598 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e5721de04ec4e0d9a2f079e38ee196e" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 38033 } } peers { permanent_uuid: "ca76b697a3ea40c1bb128ac5e5744598" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 36355 } } peers { permanent_uuid: "7b8f858dd216415bbdfab182e00a7eb1" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 38067 } }
I20260811 06:34:51.203738 27330 raft_consensus.cc:374] T ded9e07167a2429881175e61ae72f52e P ca76b697a3ea40c1bb128ac5e5744598 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:34:51.203878 27330 raft_consensus.cc:729] T ded9e07167a2429881175e61ae72f52e P ca76b697a3ea40c1bb128ac5e5744598 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca76b697a3ea40c1bb128ac5e5744598, State: Initialized, Role: FOLLOWER
I20260811 06:34:51.204563 27330 consensus_queue.cc:260] T ded9e07167a2429881175e61ae72f52e P ca76b697a3ea40c1bb128ac5e5744598 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e5721de04ec4e0d9a2f079e38ee196e" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 38033 } } peers { permanent_uuid: "ca76b697a3ea40c1bb128ac5e5744598" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 36355 } } peers { permanent_uuid: "7b8f858dd216415bbdfab182e00a7eb1" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 38067 } }
I20260811 06:34:51.207099 27297 heartbeater.cc:503] Master 127.28.239.254:44445 was elected leader, sending a full tablet report...
I20260811 06:34:51.207863 27330 ts_tablet_manager.cc:1428] T ded9e07167a2429881175e61ae72f52e P ca76b697a3ea40c1bb128ac5e5744598: Time spent starting tablet: real 0.012s user 0.009s sys 0.004s
I20260811 06:34:51.209504 27333 raft_consensus.cc:348] T ded9e07167a2429881175e61ae72f52e P 7b8f858dd216415bbdfab182e00a7eb1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e5721de04ec4e0d9a2f079e38ee196e" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 38033 } } peers { permanent_uuid: "ca76b697a3ea40c1bb128ac5e5744598" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 36355 } } peers { permanent_uuid: "7b8f858dd216415bbdfab182e00a7eb1" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 38067 } }
I20260811 06:34:51.209504 27332 raft_consensus.cc:348] T ded9e07167a2429881175e61ae72f52e P 5e5721de04ec4e0d9a2f079e38ee196e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e5721de04ec4e0d9a2f079e38ee196e" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 38033 } } peers { permanent_uuid: "ca76b697a3ea40c1bb128ac5e5744598" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 36355 } } peers { permanent_uuid: "7b8f858dd216415bbdfab182e00a7eb1" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 38067 } }
I20260811 06:34:51.209990 27333 raft_consensus.cc:374] T ded9e07167a2429881175e61ae72f52e P 7b8f858dd216415bbdfab182e00a7eb1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:34:51.209990 27332 raft_consensus.cc:374] T ded9e07167a2429881175e61ae72f52e P 5e5721de04ec4e0d9a2f079e38ee196e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:34:51.210103 27333 raft_consensus.cc:729] T ded9e07167a2429881175e61ae72f52e P 7b8f858dd216415bbdfab182e00a7eb1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b8f858dd216415bbdfab182e00a7eb1, State: Initialized, Role: FOLLOWER
I20260811 06:34:51.210115 27332 raft_consensus.cc:729] T ded9e07167a2429881175e61ae72f52e P 5e5721de04ec4e0d9a2f079e38ee196e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e5721de04ec4e0d9a2f079e38ee196e, State: Initialized, Role: FOLLOWER
I20260811 06:34:51.210749 27333 consensus_queue.cc:260] T ded9e07167a2429881175e61ae72f52e P 7b8f858dd216415bbdfab182e00a7eb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e5721de04ec4e0d9a2f079e38ee196e" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 38033 } } peers { permanent_uuid: "ca76b697a3ea40c1bb128ac5e5744598" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 36355 } } peers { permanent_uuid: "7b8f858dd216415bbdfab182e00a7eb1" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 38067 } }
I20260811 06:34:51.210742 27332 consensus_queue.cc:260] T ded9e07167a2429881175e61ae72f52e P 5e5721de04ec4e0d9a2f079e38ee196e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e5721de04ec4e0d9a2f079e38ee196e" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 38033 } } peers { permanent_uuid: "ca76b697a3ea40c1bb128ac5e5744598" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 36355 } } peers { permanent_uuid: "7b8f858dd216415bbdfab182e00a7eb1" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 38067 } }
I20260811 06:34:51.211949 27099 heartbeater.cc:503] Master 127.28.239.254:44445 was elected leader, sending a full tablet report...
I20260811 06:34:51.212841 27332 ts_tablet_manager.cc:1428] T ded9e07167a2429881175e61ae72f52e P 5e5721de04ec4e0d9a2f079e38ee196e: Time spent starting tablet: real 0.014s user 0.007s sys 0.008s
I20260811 06:34:51.214478 27333 ts_tablet_manager.cc:1428] T ded9e07167a2429881175e61ae72f52e P 7b8f858dd216415bbdfab182e00a7eb1: Time spent starting tablet: real 0.016s user 0.011s sys 0.004s
W20260811 06:34:51.335632 27100 tablet.cc:2369] T ded9e07167a2429881175e61ae72f52e P 5e5721de04ec4e0d9a2f079e38ee196e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 06:34:51.346550 27298 tablet.cc:2369] T ded9e07167a2429881175e61ae72f52e P ca76b697a3ea40c1bb128ac5e5744598: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 06:34:51.348925 26910 tablet.cc:2369] T ded9e07167a2429881175e61ae72f52e P 7b8f858dd216415bbdfab182e00a7eb1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:34:51.557938 27339 raft_consensus.cc:482] T ded9e07167a2429881175e61ae72f52e P ca76b697a3ea40c1bb128ac5e5744598 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:34:51.558293 27339 raft_consensus.cc:504] T ded9e07167a2429881175e61ae72f52e P ca76b697a3ea40c1bb128ac5e5744598 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e5721de04ec4e0d9a2f079e38ee196e" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 38033 } } peers { permanent_uuid: "ca76b697a3ea40c1bb128ac5e5744598" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 36355 } } peers { permanent_uuid: "7b8f858dd216415bbdfab182e00a7eb1" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 38067 } }
I20260811 06:34:51.560084 27339 leader_election.cc:288] T ded9e07167a2429881175e61ae72f52e P ca76b697a3ea40c1bb128ac5e5744598 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e5721de04ec4e0d9a2f079e38ee196e (127.28.239.194:38033), 7b8f858dd216415bbdfab182e00a7eb1 (127.28.239.193:38067)
I20260811 06:34:51.566776 27321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:51.560818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.195:51261 (local address 127.28.239.194:38033)
0811 06:34:51.561044 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0811 06:34:51.561051 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:34:51.561097 (+ 46us) server_negotiation.cc:407] Connection header received
0811 06:34:51.561187 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:34:51.561191 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:34:51.561265 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:34:51.561386 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:34:51.561926 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:51.562828 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:51.564888 (+ 2060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:51.565763 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:51.565931 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:51.566007 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 06:34:51.566349 (+ 342us) server_negotiation.cc:299] Negotiation successful
0811 06:34:51.566481 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260811 06:34:51.566838 27299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:51.560510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.194:38033 (local address 127.28.239.195:51261)
0811 06:34:51.560836 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:51.560874 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 06:34:51.561021 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:34:51.561436 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:34:51.561442 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:34:51.561470 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:34:51.561731 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:51.561741 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:51.562982 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:51.562989 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:51.564715 (+ 1726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:51.564728 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:51.565931 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:51.565937 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:51.566139 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:51.566189 (+ 50us) client_negotiation.cc:769] Sending connection context
0811 06:34:51.566307 (+ 118us) client_negotiation.cc:240] Negotiation successful
0811 06:34:51.566467 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"mutex_wait_us":47}
I20260811 06:34:51.566911 27327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:51.560666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.195:38741 (local address 127.28.239.193:38067)
0811 06:34:51.560916 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0811 06:34:51.560925 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:34:51.561496 (+ 571us) server_negotiation.cc:407] Connection header received
0811 06:34:51.561755 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:34:51.561760 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:34:51.561856 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:34:51.561969 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:34:51.562571 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:51.563270 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:51.565129 (+ 1859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:51.565970 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:51.566146 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:51.566222 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 06:34:51.566548 (+ 326us) server_negotiation.cc:299] Negotiation successful
0811 06:34:51.566671 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260811 06:34:51.567024 27342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:51.560591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.193:38067 (local address 127.28.239.195:38741)
0811 06:34:51.561373 (+ 782us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:51.561388 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 06:34:51.561604 (+ 216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:34:51.561985 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:34:51.561989 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:34:51.562013 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:34:51.562357 (+ 344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:51.562366 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:51.563434 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:51.563439 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:51.564930 (+ 1491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:51.564943 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:51.566146 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:51.566152 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:51.566380 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:51.566403 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 06:34:51.566506 (+ 103us) client_negotiation.cc:240] Negotiation successful
0811 06:34:51.566635 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":653,"mutex_wait_us":61,"thread_start_us":473,"threads_started":1}
I20260811 06:34:51.568157 26846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ded9e07167a2429881175e61ae72f52e" candidate_uuid: "ca76b697a3ea40c1bb128ac5e5744598" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b8f858dd216415bbdfab182e00a7eb1" is_pre_election: true
I20260811 06:34:51.568156 27029 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ded9e07167a2429881175e61ae72f52e" candidate_uuid: "ca76b697a3ea40c1bb128ac5e5744598" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5721de04ec4e0d9a2f079e38ee196e" is_pre_election: true
I20260811 06:34:51.568837 26846 raft_consensus.cc:2454] T ded9e07167a2429881175e61ae72f52e P 7b8f858dd216415bbdfab182e00a7eb1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca76b697a3ea40c1bb128ac5e5744598 in term 0.
I20260811 06:34:51.568830 27029 raft_consensus.cc:2454] T ded9e07167a2429881175e61ae72f52e P 5e5721de04ec4e0d9a2f079e38ee196e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca76b697a3ea40c1bb128ac5e5744598 in term 0.
I20260811 06:34:51.569639 27128 leader_election.cc:302] T ded9e07167a2429881175e61ae72f52e P ca76b697a3ea40c1bb128ac5e5744598 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 5e5721de04ec4e0d9a2f079e38ee196e, 7b8f858dd216415bbdfab182e00a7eb1, ca76b697a3ea40c1bb128ac5e5744598; no voters:
I20260811 06:34:51.570128 27339 raft_consensus.cc:2781] T ded9e07167a2429881175e61ae72f52e P ca76b697a3ea40c1bb128ac5e5744598 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:34:51.570286 27339 raft_consensus.cc:482] T ded9e07167a2429881175e61ae72f52e P ca76b697a3ea40c1bb128ac5e5744598 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:34:51.570431 27339 raft_consensus.cc:3037] T ded9e07167a2429881175e61ae72f52e P ca76b697a3ea40c1bb128ac5e5744598 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:34:51.572897 27339 raft_consensus.cc:504] T ded9e07167a2429881175e61ae72f52e P ca76b697a3ea40c1bb128ac5e5744598 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e5721de04ec4e0d9a2f079e38ee196e" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 38033 } } peers { permanent_uuid: "ca76b697a3ea40c1bb128ac5e5744598" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 36355 } } peers { permanent_uuid: "7b8f858dd216415bbdfab182e00a7eb1" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 38067 } }
I20260811 06:34:51.573599 27339 leader_election.cc:288] T ded9e07167a2429881175e61ae72f52e P ca76b697a3ea40c1bb128ac5e5744598 [CANDIDATE]: Term 1 election: Requested vote from peers 5e5721de04ec4e0d9a2f079e38ee196e (127.28.239.194:38033), 7b8f858dd216415bbdfab182e00a7eb1 (127.28.239.193:38067)
I20260811 06:34:51.574136 27029 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ded9e07167a2429881175e61ae72f52e" candidate_uuid: "ca76b697a3ea40c1bb128ac5e5744598" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5721de04ec4e0d9a2f079e38ee196e"
I20260811 06:34:51.574177 26846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ded9e07167a2429881175e61ae72f52e" candidate_uuid: "ca76b697a3ea40c1bb128ac5e5744598" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b8f858dd216415bbdfab182e00a7eb1"
I20260811 06:34:51.574374 27029 raft_consensus.cc:3037] T ded9e07167a2429881175e61ae72f52e P 5e5721de04ec4e0d9a2f079e38ee196e [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:34:51.574385 26846 raft_consensus.cc:3037] T ded9e07167a2429881175e61ae72f52e P 7b8f858dd216415bbdfab182e00a7eb1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:34:51.576820 26846 raft_consensus.cc:2454] T ded9e07167a2429881175e61ae72f52e P 7b8f858dd216415bbdfab182e00a7eb1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca76b697a3ea40c1bb128ac5e5744598 in term 1.
I20260811 06:34:51.577117 27029 raft_consensus.cc:2454] T ded9e07167a2429881175e61ae72f52e P 5e5721de04ec4e0d9a2f079e38ee196e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca76b697a3ea40c1bb128ac5e5744598 in term 1.
I20260811 06:34:51.577296 27122 leader_election.cc:302] T ded9e07167a2429881175e61ae72f52e P ca76b697a3ea40c1bb128ac5e5744598 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7b8f858dd216415bbdfab182e00a7eb1, ca76b697a3ea40c1bb128ac5e5744598; no voters:
I20260811 06:34:51.577672 27339 raft_consensus.cc:2781] T ded9e07167a2429881175e61ae72f52e P ca76b697a3ea40c1bb128ac5e5744598 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:34:51.578161 27339 raft_consensus.cc:686] T ded9e07167a2429881175e61ae72f52e P ca76b697a3ea40c1bb128ac5e5744598 [term 1 LEADER]: Becoming Leader. State: Replica: ca76b697a3ea40c1bb128ac5e5744598, State: Running, Role: LEADER
I20260811 06:34:51.578670 27339 consensus_queue.cc:237] T ded9e07167a2429881175e61ae72f52e P ca76b697a3ea40c1bb128ac5e5744598 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e5721de04ec4e0d9a2f079e38ee196e" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 38033 } } peers { permanent_uuid: "ca76b697a3ea40c1bb128ac5e5744598" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 36355 } } peers { permanent_uuid: "7b8f858dd216415bbdfab182e00a7eb1" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 38067 } }
I20260811 06:34:51.583796 26633 catalog_manager.cc:5495] T ded9e07167a2429881175e61ae72f52e P ca76b697a3ea40c1bb128ac5e5744598 reported cstate change: term changed from 0 to 1, leader changed from <none> to ca76b697a3ea40c1bb128ac5e5744598 (127.28.239.195). New cstate: current_term: 1 leader_uuid: "ca76b697a3ea40c1bb128ac5e5744598" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e5721de04ec4e0d9a2f079e38ee196e" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca76b697a3ea40c1bb128ac5e5744598" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 36355 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7b8f858dd216415bbdfab182e00a7eb1" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 38067 } health_report { overall_health: UNKNOWN } } }
I20260811 06:34:51.625144 27325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:51.619414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35146 (local address 127.28.239.195:36355)
0811 06:34:51.619703 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0811 06:34:51.619710 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:34:51.619741 (+ 31us) server_negotiation.cc:407] Connection header received
0811 06:34:51.619962 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:34:51.619966 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:34:51.620048 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:34:51.620174 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:34:51.620569 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:51.621307 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:51.622861 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:51.623261 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:51.623364 (+ 103us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:51.623917 (+ 553us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:34:51.624433 (+ 516us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:34:51.624579 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0811 06:34:51.624771 (+ 192us) server_negotiation.cc:299] Negotiation successful
0811 06:34:51.624840 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260811 06:34:51.634014 27029 raft_consensus.cc:1261] T ded9e07167a2429881175e61ae72f52e P 5e5721de04ec4e0d9a2f079e38ee196e [term 1 FOLLOWER]: Refusing update from remote peer ca76b697a3ea40c1bb128ac5e5744598: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:34:51.634011 26846 raft_consensus.cc:1261] T ded9e07167a2429881175e61ae72f52e P 7b8f858dd216415bbdfab182e00a7eb1 [term 1 FOLLOWER]: Refusing update from remote peer ca76b697a3ea40c1bb128ac5e5744598: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:34:51.635035 27343 consensus_queue.cc:1035] T ded9e07167a2429881175e61ae72f52e P ca76b697a3ea40c1bb128ac5e5744598 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7b8f858dd216415bbdfab182e00a7eb1" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 38067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:34:51.635385 27339 consensus_queue.cc:1035] T ded9e07167a2429881175e61ae72f52e P ca76b697a3ea40c1bb128ac5e5744598 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5721de04ec4e0d9a2f079e38ee196e" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:34:51.648557 27346 mvcc.cc:205] Tried to move back new op lower bound from 7317217655322415104 to 7317217655110479872. Current Snapshot: MvccSnapshot[applied={T|T < 7317217655322415104}]
I20260811 06:34:51.673013 26633 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:56040:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:34:51.673332 26633 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:56040:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:34:51.677204 26633 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8241ec55d42945e6ad5b8af2d3b68d78: Sending DeleteTablet for 3 replicas of tablet ded9e07167a2429881175e61ae72f52e
I20260811 06:34:51.678422 27005 tablet_service.cc:1514] Processing DeleteTablet for tablet ded9e07167a2429881175e61ae72f52e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:34:51 UTC) from {username='slave'} at 127.0.0.1:48054
I20260811 06:34:51.678884 27203 tablet_service.cc:1514] Processing DeleteTablet for tablet ded9e07167a2429881175e61ae72f52e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:34:51 UTC) from {username='slave'} at 127.0.0.1:35144
I20260811 06:34:51.679136 26815 tablet_service.cc:1514] Processing DeleteTablet for tablet ded9e07167a2429881175e61ae72f52e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:34:51 UTC) from {username='slave'} at 127.0.0.1:42036
I20260811 06:34:51.680565 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 26717
I20260811 06:34:51.691745 27364 tablet_replica.cc:330] T ded9e07167a2429881175e61ae72f52e P ca76b697a3ea40c1bb128ac5e5744598: stopping tablet replica
I20260811 06:34:51.692281 27364 raft_consensus.cc:2229] T ded9e07167a2429881175e61ae72f52e P ca76b697a3ea40c1bb128ac5e5744598 [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:34:51.693060 27364 raft_consensus.cc:2258] T ded9e07167a2429881175e61ae72f52e P ca76b697a3ea40c1bb128ac5e5744598 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:34:51.695312 27364 ts_tablet_manager.cc:1905] T ded9e07167a2429881175e61ae72f52e P ca76b697a3ea40c1bb128ac5e5744598: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:34:51.703126 27365 tablet_replica.cc:330] T ded9e07167a2429881175e61ae72f52e P 7b8f858dd216415bbdfab182e00a7eb1: stopping tablet replica
I20260811 06:34:51.703733 27327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:51.683821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42050 (local address 127.28.239.193:38067)
0811 06:34:51.684012 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 06:34:51.684020 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:34:51.694097 (+ 10077us) server_negotiation.cc:407] Connection header received
0811 06:34:51.694475 (+ 378us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:34:51.694481 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:34:51.694614 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:34:51.694705 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:34:51.695425 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:51.696698 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:51.698675 (+ 1977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:51.699358 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:51.699466 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:51.701158 (+ 1692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:34:51.701174 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:34:51.701193 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:34:51.702375 (+ 1182us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:34:51.702463 (+ 88us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:34:51.702501 (+ 38us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:34:51.702660 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 06:34:51.703132 (+ 472us) server_negotiation.cc:299] Negotiation successful
0811 06:34:51.703269 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 06:34:51.703732 27364 ts_tablet_manager.cc:1918] T ded9e07167a2429881175e61ae72f52e P ca76b697a3ea40c1bb128ac5e5744598: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:34:51.703913 27364 log.cc:1196] T ded9e07167a2429881175e61ae72f52e P ca76b697a3ea40c1bb128ac5e5744598: Deleting WAL directory at /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429755927508-29631-0/minicluster-data/ts-2/wal/wals/ded9e07167a2429881175e61ae72f52e
I20260811 06:34:51.704310 27365 raft_consensus.cc:2229] T ded9e07167a2429881175e61ae72f52e P 7b8f858dd216415bbdfab182e00a7eb1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:34:51.704756 26901 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:34:51.705359 27361 tablet_replica.cc:330] T ded9e07167a2429881175e61ae72f52e P 5e5721de04ec4e0d9a2f079e38ee196e: stopping tablet replica
I20260811 06:34:51.705693 27364 ts_tablet_manager.cc:1939] T ded9e07167a2429881175e61ae72f52e P ca76b697a3ea40c1bb128ac5e5744598: Deleting consensus metadata
I20260811 06:34:51.705864 27361 raft_consensus.cc:2229] T ded9e07167a2429881175e61ae72f52e P 5e5721de04ec4e0d9a2f079e38ee196e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:34:51.706418 27361 raft_consensus.cc:2258] T ded9e07167a2429881175e61ae72f52e P 5e5721de04ec4e0d9a2f079e38ee196e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:34:51.708015 26611 catalog_manager.cc:4841] TS ca76b697a3ea40c1bb128ac5e5744598 (127.28.239.195:36355): tablet ded9e07167a2429881175e61ae72f52e (table test-table [id=ac8479e9a04143c587f6e8bfef4847b4]) successfully deleted
I20260811 06:34:51.708963 27361 ts_tablet_manager.cc:1905] T ded9e07167a2429881175e61ae72f52e P 5e5721de04ec4e0d9a2f079e38ee196e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:34:51.717237 27361 ts_tablet_manager.cc:1918] T ded9e07167a2429881175e61ae72f52e P 5e5721de04ec4e0d9a2f079e38ee196e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:34:51.717419 27361 log.cc:1196] T ded9e07167a2429881175e61ae72f52e P 5e5721de04ec4e0d9a2f079e38ee196e: Deleting WAL directory at /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786429755927508-29631-0/minicluster-data/ts-1/wal/wals/ded9e07167a2429881175e61ae72f52e
I20260811 06:34:51.717998 27361 ts_tablet_manager.cc:1939] T ded9e07167a2429881175e61ae72f52e P 5e5721de04ec4e0d9a2f079e38ee196e: Deleting consensus metadata
I20260811 06:34:51.720152 26613 catalog_manager.cc:4841] TS 5e5721de04ec4e0d9a2f079e38ee196e (127.28.239.194:38033): tablet ded9e07167a2429881175e61ae72f52e (table test-table [id=ac8479e9a04143c587f6e8bfef4847b4]) successfully deleted
I20260811 06:34:51.799973 27365 raft_consensus.cc:2258] T ded9e07167a2429881175e61ae72f52e P 7b8f858dd216415bbdfab182e00a7eb1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:34:51.801273 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 26717
I20260811 06:34:51.825192 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 26914
W20260811 06:34:51.825258 26611 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.28.239.193:38067 (error 108)
I20260811 06:34:51.827898 27371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:51.826851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.193:38067 (local address 127.0.0.1:42054)
0811 06:34:51.827589 (+ 738us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:51.827720 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.193:38067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":578,"thread_start_us":458,"threads_started":1}
W20260811 06:34:51.828218 26611 catalog_manager.cc:4584] TS 7b8f858dd216415bbdfab182e00a7eb1 (127.28.239.193:38067): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ded9e07167a2429881175e61ae72f52e: Network error: Client connection negotiation failed: client connection to 127.28.239.193:38067: connect: Connection refused (error 111)
I20260811 06:34:51.834257 27321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:51.826550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48064 (local address 127.28.239.194:38033)
0811 06:34:51.826740 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 06:34:51.826745 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:34:51.826762 (+ 17us) server_negotiation.cc:407] Connection header received
0811 06:34:51.826824 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:34:51.826827 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:34:51.826909 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:34:51.827034 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:34:51.827582 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:51.828777 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:51.830401 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:51.831075 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:51.831252 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:51.832500 (+ 1248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:34:51.832513 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:34:51.832533 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:34:51.833417 (+ 884us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:34:51.833482 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:34:51.833509 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:34:51.833657 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 06:34:51.833763 (+ 106us) server_negotiation.cc:299] Negotiation successful
0811 06:34:51.833879 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 06:34:51.835055 27090 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:34:51.881976 27371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:51.881357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.193:38067 (local address 127.0.0.1:42062)
0811 06:34:51.881637 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:51.881757 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.193:38067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260811 06:34:51.883237 27371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:51.882809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.193:38067 (local address 127.0.0.1:42068)
0811 06:34:51.882983 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:51.883073 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.193:38067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260811 06:34:51.927176 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 26914
I20260811 06:34:51.944341 27371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:51.943599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.193:38067 (local address 127.0.0.1:42078)
0811 06:34:51.943964 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:51.944101 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.193:38067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
I20260811 06:34:51.945716 27371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:51.945224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.193:38067 (local address 127.0.0.1:42084)
0811 06:34:51.945413 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:51.945509 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.193:38067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260811 06:34:51.957437 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 27102
I20260811 06:34:51.965956 27325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:51.958429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35150 (local address 127.28.239.195:36355)
0811 06:34:51.958645 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 06:34:51.958652 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:34:51.958706 (+ 54us) server_negotiation.cc:407] Connection header received
0811 06:34:51.958778 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:34:51.958784 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:34:51.958896 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:34:51.959015 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:34:51.959396 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:51.960649 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:51.962132 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:51.962871 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:51.963024 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:51.964114 (+ 1090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:34:51.964127 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:34:51.964148 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:34:51.965022 (+ 874us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:34:51.965111 (+ 89us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:34:51.965140 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:34:51.965268 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 06:34:51.965400 (+ 132us) server_negotiation.cc:299] Negotiation successful
0811 06:34:51.965506 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 06:34:51.966952 27288 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:34:52.026434 27371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:52.025864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.193:38067 (local address 127.0.0.1:42088)
0811 06:34:52.026107 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:52.026222 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.193:38067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260811 06:34:52.027667 27371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:52.027175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.193:38067 (local address 127.0.0.1:42104)
0811 06:34:52.027407 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:52.027500 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.193:38067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122}
I20260811 06:34:52.091403 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 27102
I20260811 06:34:52.117434 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 26591
I20260811 06:34:52.118635 26677 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:34:52.227883 27371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:52.227245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.193:38067 (local address 127.0.0.1:42106)
0811 06:34:52.227486 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:52.227650 (+ 164us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.193:38067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260811 06:34:52.228246 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 26591
2026-08-11T06:34:52Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (3270 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:34:52 dist-test-slave-gkw7 krb5kdc[27379](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:34:52 dist-test-slave-gkw7 krb5kdc[27379](info): set up 2 sockets
Aug 11 06:34:52 dist-test-slave-gkw7 krb5kdc[27379](info): commencing operation
krb5kdc: starting...
W20260811 06:34:53.586838 29631 mini_kdc.cc:121] Time spent starting KDC: real 1.303s user 0.002s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/krb5kdc/test-user.keytab.
Aug 11 06:34:53 dist-test-slave-gkw7 krb5kdc[27379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430093, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:34:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:34:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.239.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.239.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.239.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab.
Entry for principal kudu/127.28.239.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab.
I20260811 06:34:53.921615 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:38423
--webserver_interface=127.28.239.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab
--principal=kudu/127.28.239.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:36473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.239.254:38423
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:34:54.277331 27399 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:34:54.277704 27399 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:34:54.277805 27399 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:34:54.285235 27399 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:34:54.285337 27399 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:34:54.285385 27399 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:34:54.285435 27399 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:34:54.285480 27399 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:34:54.295181 27399 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:36473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.239.254:38423
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab
--principal=kudu/127.28.239.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:38423
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.239.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:34:54.296453 27399 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:34:54.297731 27399 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:34:54.310297 27407 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:34:54.310328 27408 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:34:54.310467 27412 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:34:54.311460 27399 server_base.cc:1034] running on GCE node
I20260811 06:34:54.312320 27399 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:34:54.313892 27399 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:34:54.315155 27399 hybrid_clock.cc:648] HybridClock initialized: now 1786430094315102 us; error 29 us; skew 500 ppm
I20260811 06:34:54.315563 27399 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:34:54 dist-test-slave-gkw7 krb5kdc[27379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430094, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:34:54.323503 27399 init.cc:377] Logged in from keytab as kudu/127.28.239.254@KRBTEST.COM (short username kudu)
I20260811 06:34:54.325289 27399 webserver.cc:460] Webserver started at http://127.28.239.254:44345/ using document root <none> and password file <none>
I20260811 06:34:54.326135 27399 fs_manager.cc:359] Metadata directory not provided
I20260811 06:34:54.326306 27399 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:34:54.326665 27399 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:34:54.330124 27399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/minicluster-data/master-0/data/instance:
uuid: "8e691a02f86a47388427deb0843c8967"
format_stamp: "Formatted at 2026-08-11 06:34:54 on dist-test-slave-gkw7"
server_key: "f1e47d805be3e9adff7186d52201b5b3"
server_key_iv: "9c34cdca986ec28f2ad7f8b1ca3dd9d9"
server_key_version: "encryptionkey@0"
I20260811 06:34:54.330967 27399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/minicluster-data/master-0/wal/instance:
uuid: "8e691a02f86a47388427deb0843c8967"
format_stamp: "Formatted at 2026-08-11 06:34:54 on dist-test-slave-gkw7"
server_key: "f1e47d805be3e9adff7186d52201b5b3"
server_key_iv: "9c34cdca986ec28f2ad7f8b1ca3dd9d9"
server_key_version: "encryptionkey@0"
I20260811 06:34:54.337018 27399 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260811 06:34:54.340962 27423 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:34:54.342592 27399 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 06:34:54.342830 27399 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/minicluster-data/master-0/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/minicluster-data/master-0/wal
uuid: "8e691a02f86a47388427deb0843c8967"
format_stamp: "Formatted at 2026-08-11 06:34:54 on dist-test-slave-gkw7"
server_key: "f1e47d805be3e9adff7186d52201b5b3"
server_key_iv: "9c34cdca986ec28f2ad7f8b1ca3dd9d9"
server_key_version: "encryptionkey@0"
I20260811 06:34:54.343060 27399 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:34:54.361068 27399 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:34:54.365761 27399 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:34:54.366101 27399 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:34:54.387286 27399 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.254:38423
I20260811 06:34:54.387295 27498 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.254:38423 every 8 connection(s)
I20260811 06:34:54.388870 27399 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
I20260811 06:34:54.392807 27499 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:34:54.396488 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 27399
I20260811 06:34:54.396697 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/minicluster-data/master-0/wal/instance
I20260811 06:34:54.397166 29631 external_mini_cluster.cc:1428] Setting key dbce57aa71c9c387d55bacff082b9f99
I20260811 06:34:54.406261 27499 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e691a02f86a47388427deb0843c8967: Bootstrap starting.
I20260811 06:34:54.410411 27499 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8e691a02f86a47388427deb0843c8967: Neither blocks nor log segments found. Creating new log.
I20260811 06:34:54.411839 27499 log.cc:824] T 00000000000000000000000000000000 P 8e691a02f86a47388427deb0843c8967: Log is configured to *not* fsync() on all Append() calls
Aug 11 06:34:54 dist-test-slave-gkw7 krb5kdc[27379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430093, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:34:54.415038 27499 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e691a02f86a47388427deb0843c8967: No bootstrap required, opened a new log
I20260811 06:34:54.422299 27506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:54.399176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59282 (local address 127.28.239.254:38423)
0811 06:34:54.400030 (+ 854us) server_negotiation.cc:206] Beginning negotiation
0811 06:34:54.400042 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 06:34:54.400086 (+ 44us) server_negotiation.cc:407] Connection header received
0811 06:34:54.401924 (+ 1838us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:34:54.401950 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:34:54.402387 (+ 437us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:34:54.402726 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:34:54.404712 (+ 1986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:54.406194 (+ 1482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:54.408076 (+ 1882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:54.408789 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:54.408926 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:54.414435 (+ 5509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:34:54.414443 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:34:54.414459 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:34:54.414496 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:34:54.417884 (+ 3388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:34:54.418618 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:34:54.418629 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:34:54.418636 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:34:54.418806 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:34:54.419238 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:34:54.419249 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:34:54.419251 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:34:54.419827 (+ 576us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:34:54.420162 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0811 06:34:54.420686 (+ 524us) server_negotiation.cc:299] Negotiation successful
0811 06:34:54.421127 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":627,"thread_start_us":383,"threads_started":1}
I20260811 06:34:54.423199 27499 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8e691a02f86a47388427deb0843c8967 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e691a02f86a47388427deb0843c8967" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 38423 } }
I20260811 06:34:54.423640 27499 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8e691a02f86a47388427deb0843c8967 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:34:54.423767 27499 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8e691a02f86a47388427deb0843c8967 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e691a02f86a47388427deb0843c8967, State: Initialized, Role: FOLLOWER
I20260811 06:34:54.424463 27499 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8e691a02f86a47388427deb0843c8967 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e691a02f86a47388427deb0843c8967" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 38423 } }
I20260811 06:34:54.424719 27499 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8e691a02f86a47388427deb0843c8967 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:34:54.424875 27499 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8e691a02f86a47388427deb0843c8967 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:34:54.425074 27499 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8e691a02f86a47388427deb0843c8967 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:34:54.427556 27499 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8e691a02f86a47388427deb0843c8967 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e691a02f86a47388427deb0843c8967" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 38423 } }
I20260811 06:34:54.428089 27499 leader_election.cc:302] T 00000000000000000000000000000000 P 8e691a02f86a47388427deb0843c8967 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8e691a02f86a47388427deb0843c8967; no voters:
I20260811 06:34:54.428746 27499 leader_election.cc:288] T 00000000000000000000000000000000 P 8e691a02f86a47388427deb0843c8967 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:34:54.428874 27508 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8e691a02f86a47388427deb0843c8967 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:34:54.429605 27508 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8e691a02f86a47388427deb0843c8967 [term 1 LEADER]: Becoming Leader. State: Replica: 8e691a02f86a47388427deb0843c8967, State: Running, Role: LEADER
I20260811 06:34:54.430267 27508 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8e691a02f86a47388427deb0843c8967 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e691a02f86a47388427deb0843c8967" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 38423 } }
I20260811 06:34:54.431254 27499 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8e691a02f86a47388427deb0843c8967 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:34:54.433105 27509 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e691a02f86a47388427deb0843c8967 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8e691a02f86a47388427deb0843c8967" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e691a02f86a47388427deb0843c8967" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 38423 } } }
I20260811 06:34:54.433514 27509 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e691a02f86a47388427deb0843c8967 [sys.catalog]: This master's current role is: LEADER
I20260811 06:34:54.435158 27519 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:34:54.435180 27512 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e691a02f86a47388427deb0843c8967 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8e691a02f86a47388427deb0843c8967. Latest consensus state: current_term: 1 leader_uuid: "8e691a02f86a47388427deb0843c8967" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e691a02f86a47388427deb0843c8967" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 38423 } } }
I20260811 06:34:54.435499 27512 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e691a02f86a47388427deb0843c8967 [sys.catalog]: This master's current role is: LEADER
I20260811 06:34:54.441231 27519 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:34:54.451769 27519 catalog_manager.cc:1347] Generated new cluster ID: d018625ceab845de95f87ff38f2205d5
I20260811 06:34:54.451908 27519 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:34:54.467060 27519 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:34:54.468963 27519 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:34:54.483330 27519 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8e691a02f86a47388427deb0843c8967: Generated new TSK 0
I20260811 06:34:54.484237 27519 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.239.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.239.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.239.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.193.keytab.
Entry for principal kudu/127.28.239.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.193.keytab.
I20260811 06:34:54.568663 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.193:0
--local_ip_for_outbound_sockets=127.28.239.193
--webserver_interface=127.28.239.193
--webserver_port=0
--tserver_master_addrs=127.28.239.254:38423
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.193.keytab
--principal=kudu/127.28.239.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:36473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:34:54.924378 27535 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:34:54.924790 27535 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:34:54.924965 27535 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:34:54.932466 27535 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:34:54.932614 27535 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:34:54.932790 27535 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.193
I20260811 06:34:54.943455 27535 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:36473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.193.keytab
--principal=kudu/127.28.239.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.239.193
--webserver_port=0
--tserver_master_addrs=127.28.239.254:38423
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.193
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:34:54.944857 27535 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:34:54.946411 27535 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:34:54.960871 27540 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:34:54.961011 27541 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:34:54.961243 27544 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:34:54.962261 27535 server_base.cc:1034] running on GCE node
I20260811 06:34:54.963090 27535 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:34:54.964191 27535 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:34:54.965432 27535 hybrid_clock.cc:648] HybridClock initialized: now 1786430094965371 us; error 46 us; skew 500 ppm
I20260811 06:34:54.965845 27535 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:34:54 dist-test-slave-gkw7 krb5kdc[27379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430094, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:34:54.969606 27535 init.cc:377] Logged in from keytab as kudu/127.28.239.193@KRBTEST.COM (short username kudu)
I20260811 06:34:54.971197 27535 webserver.cc:460] Webserver started at http://127.28.239.193:41761/ using document root <none> and password file <none>
I20260811 06:34:54.971967 27535 fs_manager.cc:359] Metadata directory not provided
I20260811 06:34:54.972126 27535 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:34:54.972491 27535 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:34:54.976148 27535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/minicluster-data/ts-0/data/instance:
uuid: "38dbe8736e09473db77120b314f24838"
format_stamp: "Formatted at 2026-08-11 06:34:54 on dist-test-slave-gkw7"
server_key: "1afd0a0179c108b65a42184817644db9"
server_key_iv: "850528724db4492ac5415c5c77d69340"
server_key_version: "encryptionkey@0"
I20260811 06:34:54.977053 27535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/minicluster-data/ts-0/wal/instance:
uuid: "38dbe8736e09473db77120b314f24838"
format_stamp: "Formatted at 2026-08-11 06:34:54 on dist-test-slave-gkw7"
server_key: "1afd0a0179c108b65a42184817644db9"
server_key_iv: "850528724db4492ac5415c5c77d69340"
server_key_version: "encryptionkey@0"
I20260811 06:34:54.983669 27535 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.003s
I20260811 06:34:54.987411 27554 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:34:54.988801 27535 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 06:34:54.988989 27535 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/minicluster-data/ts-0/wal
uuid: "38dbe8736e09473db77120b314f24838"
format_stamp: "Formatted at 2026-08-11 06:34:54 on dist-test-slave-gkw7"
server_key: "1afd0a0179c108b65a42184817644db9"
server_key_iv: "850528724db4492ac5415c5c77d69340"
server_key_version: "encryptionkey@0"
I20260811 06:34:54.989204 27535 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:34:55.013751 27535 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:34:55.019335 27535 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:34:55.019683 27535 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:34:55.021073 27535 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:34:55.022634 27535 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:34:55.022755 27535 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:34:55.022917 27535 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:34:55.022990 27535 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:34:55.060025 27535 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.193:34063
I20260811 06:34:55.060128 27717 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.193:34063 every 8 connection(s)
I20260811 06:34:55.061467 27535 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
I20260811 06:34:55.063985 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 27535
I20260811 06:34:55.064191 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/minicluster-data/ts-0/wal/instance
I20260811 06:34:55.064746 29631 external_mini_cluster.cc:1428] Setting key 30d7202b53eb229c706832623d4e6793
Aug 11 06:34:55 dist-test-slave-gkw7 krb5kdc[27379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430094, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.193@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:34:55.088268 27720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:55.064760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.193:41163 (local address 127.28.239.254:38423)
0811 06:34:55.065313 (+ 553us) server_negotiation.cc:206] Beginning negotiation
0811 06:34:55.065320 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:34:55.068806 (+ 3486us) server_negotiation.cc:407] Connection header received
0811 06:34:55.068858 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:34:55.068862 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:34:55.068932 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:34:55.069017 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:34:55.071519 (+ 2502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:55.072406 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:55.075301 (+ 2895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:55.075739 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:55.075898 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:55.080760 (+ 4862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:34:55.080765 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:34:55.080769 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:34:55.080806 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:34:55.084409 (+ 3603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:34:55.085602 (+ 1193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:34:55.085609 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:34:55.085613 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:34:55.085760 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:34:55.086335 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:34:55.086338 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:34:55.086340 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:34:55.086612 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:34:55.086726 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 06:34:55.087771 (+ 1045us) server_negotiation.cc:299] Negotiation successful
0811 06:34:55.087969 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":271,"threads_started":1}
I20260811 06:34:55.090333 27721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:55.065699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:38423 (local address 127.28.239.193:41163)
0811 06:34:55.066635 (+ 936us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:55.066706 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0811 06:34:55.068508 (+ 1802us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:34:55.069364 (+ 856us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:34:55.069381 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:34:55.070313 (+ 932us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:34:55.071271 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:55.071308 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:55.072640 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:55.072649 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:55.075105 (+ 2456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:55.075125 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:55.076044 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:55.076056 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:55.076430 (+ 374us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:55.077328 (+ 898us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:34:55.077368 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:34:55.080465 (+ 3097us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:34:55.084577 (+ 4112us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:34:55.084589 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:34:55.084613 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:34:55.085369 (+ 756us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:34:55.085885 (+ 516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:34:55.085893 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:34:55.085901 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:34:55.086195 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:34:55.086752 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:34:55.086764 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:34:55.087310 (+ 546us) client_negotiation.cc:769] Sending connection context
0811 06:34:55.087688 (+ 378us) client_negotiation.cc:240] Negotiation successful
0811 06:34:55.088556 (+ 868us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":622,"thread_start_us":356,"threads_started":1}
I20260811 06:34:55.092185 27718 heartbeater.cc:347] Connected to a master server at 127.28.239.254:38423
I20260811 06:34:55.092588 27718 heartbeater.cc:464] Registering TS with master...
I20260811 06:34:55.093473 27718 heartbeater.cc:511] Master 127.28.239.254:38423 requested a full tablet report, sending...
I20260811 06:34:55.095844 27446 ts_manager.cc:194] Registered new tserver with Master: 38dbe8736e09473db77120b314f24838 (127.28.239.193:34063)
I20260811 06:34:55.097371 27446 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.239.193@KRBTEST.COM'} at 127.28.239.193:41163
WARNING: no policy specified for kudu/127.28.239.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.239.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.239.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.194.keytab.
Entry for principal kudu/127.28.239.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.194.keytab.
I20260811 06:34:55.146106 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.194:0
--local_ip_for_outbound_sockets=127.28.239.194
--webserver_interface=127.28.239.194
--webserver_port=0
--tserver_master_addrs=127.28.239.254:38423
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.194.keytab
--principal=kudu/127.28.239.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:36473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:34:55.509172 27724 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:34:55.509513 27724 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:34:55.509740 27724 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:34:55.517093 27724 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:34:55.517268 27724 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:34:55.517452 27724 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.194
I20260811 06:34:55.527698 27724 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:36473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.194.keytab
--principal=kudu/127.28.239.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.239.194
--webserver_port=0
--tserver_master_addrs=127.28.239.254:38423
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.194
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:34:55.529115 27724 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:34:55.530599 27724 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:34:55.545639 27736 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:34:55.545817 27737 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:34:55.545619 27739 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:34:55.547267 27724 server_base.cc:1034] running on GCE node
I20260811 06:34:55.548148 27724 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:34:55.549232 27724 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:34:55.550482 27724 hybrid_clock.cc:648] HybridClock initialized: now 1786430095550414 us; error 60 us; skew 500 ppm
I20260811 06:34:55.550848 27724 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:34:55 dist-test-slave-gkw7 krb5kdc[27379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430095, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:34:55.554147 27724 init.cc:377] Logged in from keytab as kudu/127.28.239.194@KRBTEST.COM (short username kudu)
I20260811 06:34:55.555883 27724 webserver.cc:460] Webserver started at http://127.28.239.194:34045/ using document root <none> and password file <none>
I20260811 06:34:55.556610 27724 fs_manager.cc:359] Metadata directory not provided
I20260811 06:34:55.556717 27724 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:34:55.556993 27724 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:34:55.560098 27724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/minicluster-data/ts-1/data/instance:
uuid: "ce9c13504c3d4110b12f825ee9654b08"
format_stamp: "Formatted at 2026-08-11 06:34:55 on dist-test-slave-gkw7"
server_key: "8a374eafcc5e70bcfdce8a874cc6132d"
server_key_iv: "a9d4b8112e0ecaf49e92d5b4174f2b79"
server_key_version: "encryptionkey@0"
I20260811 06:34:55.560820 27724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/minicluster-data/ts-1/wal/instance:
uuid: "ce9c13504c3d4110b12f825ee9654b08"
format_stamp: "Formatted at 2026-08-11 06:34:55 on dist-test-slave-gkw7"
server_key: "8a374eafcc5e70bcfdce8a874cc6132d"
server_key_iv: "a9d4b8112e0ecaf49e92d5b4174f2b79"
server_key_version: "encryptionkey@0"
I20260811 06:34:55.566105 27724 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260811 06:34:55.569701 27754 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:34:55.571071 27724 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 06:34:55.571280 27724 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/minicluster-data/ts-1/wal
uuid: "ce9c13504c3d4110b12f825ee9654b08"
format_stamp: "Formatted at 2026-08-11 06:34:55 on dist-test-slave-gkw7"
server_key: "8a374eafcc5e70bcfdce8a874cc6132d"
server_key_iv: "a9d4b8112e0ecaf49e92d5b4174f2b79"
server_key_version: "encryptionkey@0"
I20260811 06:34:55.571503 27724 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:34:55.612955 27724 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:34:55.617522 27724 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:34:55.617872 27724 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:34:55.618870 27724 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:34:55.620281 27724 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:34:55.620388 27724 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:34:55.620537 27724 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:34:55.620616 27724 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:34:55.655316 27724 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.194:37571
I20260811 06:34:55.655408 27920 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.194:37571 every 8 connection(s)
I20260811 06:34:55.656682 27724 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
I20260811 06:34:55.660236 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 27724
I20260811 06:34:55.660449 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/minicluster-data/ts-1/wal/instance
I20260811 06:34:55.660938 29631 external_mini_cluster.cc:1428] Setting key a01d6485e6745a96d7e4a0ad66ec3907
Aug 11 06:34:55 dist-test-slave-gkw7 krb5kdc[27379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430095, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.194@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:34:55.679623 27923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:55.659609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.194:46315 (local address 127.28.239.254:38423)
0811 06:34:55.660107 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0811 06:34:55.660124 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 06:34:55.661284 (+ 1160us) server_negotiation.cc:407] Connection header received
0811 06:34:55.663450 (+ 2166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:34:55.663456 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:34:55.663541 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:34:55.663666 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:34:55.665180 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:55.666401 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:55.667935 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:55.668634 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:55.668791 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:55.672288 (+ 3497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:34:55.672297 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:34:55.672305 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:34:55.672352 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:34:55.676226 (+ 3874us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:34:55.677050 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:34:55.677058 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:34:55.677065 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:34:55.677240 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:34:55.677845 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:34:55.677849 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:34:55.677851 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:34:55.678139 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:34:55.678256 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 06:34:55.679185 (+ 929us) server_negotiation.cc:299] Negotiation successful
0811 06:34:55.679367 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":246,"threads_started":1}
I20260811 06:34:55.681218 27926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:55.660212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:38423 (local address 127.28.239.194:46315)
0811 06:34:55.661073 (+ 861us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:55.661136 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 06:34:55.663161 (+ 2025us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:34:55.663866 (+ 705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:34:55.663873 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:34:55.664432 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:34:55.664998 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:55.665016 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:55.666565 (+ 1549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:55.666570 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:55.667786 (+ 1216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:55.667798 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:55.668849 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:55.668854 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:55.669064 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:55.669848 (+ 784us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:34:55.669871 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:34:55.672031 (+ 2160us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:34:55.676396 (+ 4365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:34:55.676422 (+ 26us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:34:55.676440 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:34:55.676914 (+ 474us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:34:55.677384 (+ 470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:34:55.677391 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:34:55.677395 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:34:55.677716 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:34:55.678278 (+ 562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:34:55.678287 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:34:55.678782 (+ 495us) client_negotiation.cc:769] Sending connection context
0811 06:34:55.679100 (+ 318us) client_negotiation.cc:240] Negotiation successful
0811 06:34:55.679699 (+ 599us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":584,"thread_start_us":391,"threads_started":1}
I20260811 06:34:55.682848 27921 heartbeater.cc:347] Connected to a master server at 127.28.239.254:38423
I20260811 06:34:55.683202 27921 heartbeater.cc:464] Registering TS with master...
I20260811 06:34:55.684026 27921 heartbeater.cc:511] Master 127.28.239.254:38423 requested a full tablet report, sending...
I20260811 06:34:55.685503 27446 ts_manager.cc:194] Registered new tserver with Master: ce9c13504c3d4110b12f825ee9654b08 (127.28.239.194:37571)
I20260811 06:34:55.686432 27446 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.239.194@KRBTEST.COM'} at 127.28.239.194:46315
WARNING: no policy specified for kudu/127.28.239.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.239.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.239.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.195.keytab.
Entry for principal kudu/127.28.239.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.195.keytab.
I20260811 06:34:55.736975 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.195:0
--local_ip_for_outbound_sockets=127.28.239.195
--webserver_interface=127.28.239.195
--webserver_port=0
--tserver_master_addrs=127.28.239.254:38423
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.195.keytab
--principal=kudu/127.28.239.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:36473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 06:34:56.101425 27718 heartbeater.cc:503] Master 127.28.239.254:38423 was elected leader, sending a full tablet report...
W20260811 06:34:56.108394 27929 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:34:56.108860 27929 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:34:56.109102 27929 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:34:56.116967 27929 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:34:56.117115 27929 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:34:56.117245 27929 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.195
I20260811 06:34:56.127789 27929 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:36473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.195.keytab
--principal=kudu/127.28.239.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.239.195
--webserver_port=0
--tserver_master_addrs=127.28.239.254:38423
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.195
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:34:56.129302 27929 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:34:56.130769 27929 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:34:56.145462 27935 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:34:56.145481 27938 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:34:56.145519 27936 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:34:56.146639 27929 server_base.cc:1034] running on GCE node
I20260811 06:34:56.147390 27929 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:34:56.148474 27929 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:34:56.149711 27929 hybrid_clock.cc:648] HybridClock initialized: now 1786430096149658 us; error 49 us; skew 500 ppm
I20260811 06:34:56.150092 27929 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:34:56 dist-test-slave-gkw7 krb5kdc[27379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430096, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:34:56.153749 27929 init.cc:377] Logged in from keytab as kudu/127.28.239.195@KRBTEST.COM (short username kudu)
I20260811 06:34:56.155383 27929 webserver.cc:460] Webserver started at http://127.28.239.195:46093/ using document root <none> and password file <none>
I20260811 06:34:56.156105 27929 fs_manager.cc:359] Metadata directory not provided
I20260811 06:34:56.156219 27929 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:34:56.156518 27929 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:34:56.160058 27929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/minicluster-data/ts-2/data/instance:
uuid: "5afe3e6f759e42e2b64a820f20656205"
format_stamp: "Formatted at 2026-08-11 06:34:56 on dist-test-slave-gkw7"
server_key: "949ad8a29af0f0249ce48a91a50c2276"
server_key_iv: "c6f459035d2ca54c479c713310c8d44d"
server_key_version: "encryptionkey@0"
I20260811 06:34:56.160869 27929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/minicluster-data/ts-2/wal/instance:
uuid: "5afe3e6f759e42e2b64a820f20656205"
format_stamp: "Formatted at 2026-08-11 06:34:56 on dist-test-slave-gkw7"
server_key: "949ad8a29af0f0249ce48a91a50c2276"
server_key_iv: "c6f459035d2ca54c479c713310c8d44d"
server_key_version: "encryptionkey@0"
I20260811 06:34:56.166872 27929 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.005s
I20260811 06:34:56.171185 27948 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:34:56.172729 27929 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.002s
I20260811 06:34:56.172881 27929 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/minicluster-data/ts-2/wal
uuid: "5afe3e6f759e42e2b64a820f20656205"
format_stamp: "Formatted at 2026-08-11 06:34:56 on dist-test-slave-gkw7"
server_key: "949ad8a29af0f0249ce48a91a50c2276"
server_key_iv: "c6f459035d2ca54c479c713310c8d44d"
server_key_version: "encryptionkey@0"
I20260811 06:34:56.173086 27929 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:34:56.203616 27929 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:34:56.208858 27929 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:34:56.209157 27929 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:34:56.210302 27929 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:34:56.211861 27929 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:34:56.211941 27929 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:34:56.212033 27929 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:34:56.212092 27929 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:34:56.250248 27929 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.195:34521
I20260811 06:34:56.250288 28132 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.195:34521 every 8 connection(s)
I20260811 06:34:56.251773 27929 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
I20260811 06:34:56.261886 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 27929
I20260811 06:34:56.262137 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/minicluster-data/ts-2/wal/instance
I20260811 06:34:56.262800 29631 external_mini_cluster.cc:1428] Setting key beb0f288b0dada0eb6cea0bb8f26085c
Aug 11 06:34:56 dist-test-slave-gkw7 krb5kdc[27379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430096, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.195@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:34:56.272763 28136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:56.254549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.195:40629 (local address 127.28.239.254:38423)
0811 06:34:56.255040 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0811 06:34:56.255047 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:34:56.256115 (+ 1068us) server_negotiation.cc:407] Connection header received
0811 06:34:56.258024 (+ 1909us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:34:56.258029 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:34:56.258134 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:34:56.258243 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:34:56.259820 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:56.260694 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:56.262445 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:56.263139 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:56.263315 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:56.266908 (+ 3593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:34:56.266914 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:34:56.266917 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:34:56.266953 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:34:56.269778 (+ 2825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:34:56.270519 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:34:56.270523 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:34:56.270525 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:34:56.270665 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:34:56.271054 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:34:56.271058 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:34:56.271067 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:34:56.271356 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:34:56.271482 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 06:34:56.272180 (+ 698us) server_negotiation.cc:299] Negotiation successful
0811 06:34:56.272405 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":225,"threads_started":1}
I20260811 06:34:56.273801 28138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:56.255201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:38423 (local address 127.28.239.195:40629)
0811 06:34:56.255935 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:56.255982 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 06:34:56.257729 (+ 1747us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:34:56.258453 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:34:56.258461 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:34:56.259007 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:34:56.259646 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:56.259665 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:56.260840 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:56.260844 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:56.262284 (+ 1440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:56.262303 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:56.263408 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:56.263417 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:56.263707 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:56.264447 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:34:56.264470 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:34:56.266719 (+ 2249us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:34:56.269920 (+ 3201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:34:56.269928 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:34:56.269941 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:34:56.270385 (+ 444us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:34:56.270765 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:34:56.270768 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:34:56.270771 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:34:56.270949 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:34:56.271467 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:34:56.271473 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:34:56.271792 (+ 319us) client_negotiation.cc:769] Sending connection context
0811 06:34:56.272021 (+ 229us) client_negotiation.cc:240] Negotiation successful
0811 06:34:56.272453 (+ 432us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":284,"threads_started":1}
I20260811 06:34:56.275342 28133 heartbeater.cc:347] Connected to a master server at 127.28.239.254:38423
I20260811 06:34:56.275663 28133 heartbeater.cc:464] Registering TS with master...
I20260811 06:34:56.276494 28133 heartbeater.cc:511] Master 127.28.239.254:38423 requested a full tablet report, sending...
I20260811 06:34:56.278131 27446 ts_manager.cc:194] Registered new tserver with Master: 5afe3e6f759e42e2b64a820f20656205 (127.28.239.195:34521)
I20260811 06:34:56.278753 29631 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 06:34:56.279083 27446 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.239.195@KRBTEST.COM'} at 127.28.239.195:40629
I20260811 06:34:56.296988 28136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:56.283358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36586 (local address 127.28.239.254:38423)
0811 06:34:56.283601 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0811 06:34:56.283610 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:34:56.284073 (+ 463us) server_negotiation.cc:407] Connection header received
0811 06:34:56.284267 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:34:56.284293 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:34:56.284371 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:34:56.284479 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:34:56.286282 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:56.287007 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:56.288290 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:56.288720 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:56.288861 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:56.291337 (+ 2476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:34:56.291343 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:34:56.291348 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:34:56.291389 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:34:56.294271 (+ 2882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:34:56.294920 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:34:56.294926 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:34:56.294930 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:34:56.295059 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:34:56.295455 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:34:56.295461 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:34:56.295468 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:34:56.295816 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:34:56.296076 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0811 06:34:56.296416 (+ 340us) server_negotiation.cc:299] Negotiation successful
0811 06:34:56.296659 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 06:34:56.301182 27446 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36586:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:34:56.308229 27446 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:34:56.346928 28159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:56.322208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54584 (local address 127.28.239.194:37571)
0811 06:34:56.332786 (+ 10578us) server_negotiation.cc:206] Beginning negotiation
0811 06:34:56.332797 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 06:34:56.334888 (+ 2091us) server_negotiation.cc:407] Connection header received
0811 06:34:56.335286 (+ 398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:34:56.335299 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:34:56.335686 (+ 387us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:34:56.335915 (+ 229us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:34:56.337322 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:56.340775 (+ 3453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:56.343861 (+ 3086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:56.345438 (+ 1577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:56.345683 (+ 245us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:56.345885 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0811 06:34:56.346321 (+ 436us) server_negotiation.cc:299] Negotiation successful
0811 06:34:56.346487 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10298,"thread_start_us":10593,"threads_started":1}
I20260811 06:34:56.347362 28153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:56.321840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42856 (local address 127.28.239.195:34521)
0811 06:34:56.337633 (+ 15793us) server_negotiation.cc:206] Beginning negotiation
0811 06:34:56.337646 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 06:34:56.337693 (+ 47us) server_negotiation.cc:407] Connection header received
0811 06:34:56.337814 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:34:56.337825 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:34:56.338099 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:34:56.339572 (+ 1473us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:34:56.339946 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:56.341583 (+ 1637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:56.344287 (+ 2704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:56.345862 (+ 1575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:56.346099 (+ 237us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:56.346292 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0811 06:34:56.346800 (+ 508us) server_negotiation.cc:299] Negotiation successful
0811 06:34:56.346986 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15512,"thread_start_us":16477,"threads_started":1}
I20260811 06:34:56.347692 28158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:56.321776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.195:34521 (local address 127.0.0.1:42856)
0811 06:34:56.333645 (+ 11869us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:56.333690 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 06:34:56.333907 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:34:56.338673 (+ 4766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:34:56.338681 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:34:56.338720 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:34:56.339312 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:56.339328 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:56.341736 (+ 2408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:56.341743 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:56.344081 (+ 2338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:56.344096 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:56.346112 (+ 2016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:56.346119 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:56.346363 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:56.346403 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 06:34:56.346512 (+ 109us) client_negotiation.cc:240] Negotiation successful
0811 06:34:56.346654 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11562,"mutex_wait_us":57,"thread_start_us":12496,"threads_started":1}
I20260811 06:34:56.347815 28148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:56.322116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.194:37571 (local address 127.0.0.1:54584)
0811 06:34:56.334709 (+ 12593us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:56.334736 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 06:34:56.335028 (+ 292us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:34:56.336371 (+ 1343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:34:56.336384 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:34:56.336443 (+ 59us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:34:56.337051 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:56.337071 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:56.340982 (+ 3911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:56.340987 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:56.343669 (+ 2682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:56.343683 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:56.345804 (+ 2121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:56.345811 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:56.346026 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:56.346095 (+ 69us) client_negotiation.cc:769] Sending connection context
0811 06:34:56.346254 (+ 159us) client_negotiation.cc:240] Negotiation successful
0811 06:34:56.346448 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12393,"mutex_wait_us":7,"thread_start_us":11081,"threads_started":1}
I20260811 06:34:56.350376 28162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:56.325108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.193:34063 (local address 127.0.0.1:44630)
0811 06:34:56.340732 (+ 15624us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:56.340750 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 06:34:56.340855 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:34:56.342095 (+ 1240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:34:56.342101 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:34:56.342134 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:34:56.342706 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:56.342722 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:56.344619 (+ 1897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:56.344637 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:56.346947 (+ 2310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:56.346963 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:56.349388 (+ 2425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:56.349396 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:56.349634 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:56.349673 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 06:34:56.349783 (+ 110us) client_negotiation.cc:240] Negotiation successful
0811 06:34:56.349924 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15490,"mutex_wait_us":125,"thread_start_us":15348,"threads_started":1}
I20260811 06:34:56.351327 28163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:56.325308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44630 (local address 127.28.239.193:34063)
0811 06:34:56.341372 (+ 16064us) server_negotiation.cc:206] Beginning negotiation
0811 06:34:56.341421 (+ 49us) server_negotiation.cc:399] Waiting for connection header
0811 06:34:56.341452 (+ 31us) server_negotiation.cc:407] Connection header received
0811 06:34:56.341575 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:34:56.341582 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:34:56.341863 (+ 281us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:34:56.342069 (+ 206us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:34:56.342917 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:56.344449 (+ 1532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:56.347162 (+ 2713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:56.348776 (+ 1614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:56.349012 (+ 236us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:56.349199 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0811 06:34:56.350774 (+ 1575us) server_negotiation.cc:299] Negotiation successful
0811 06:34:56.350963 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15907,"thread_start_us":15653,"threads_started":1}
I20260811 06:34:56.351939 27820 tablet_service.cc:1467] Processing CreateTablet for tablet 08042cc961ff4aae99fd31c466823296 (DEFAULT_TABLE table=test-table [id=eee78824156448ffb2cf43fe2309a2f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:34:56.352118 28024 tablet_service.cc:1467] Processing CreateTablet for tablet 08042cc961ff4aae99fd31c466823296 (DEFAULT_TABLE table=test-table [id=eee78824156448ffb2cf43fe2309a2f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:34:56.353461 27820 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08042cc961ff4aae99fd31c466823296. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:34:56.353497 28024 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08042cc961ff4aae99fd31c466823296. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:34:56.355216 27622 tablet_service.cc:1467] Processing CreateTablet for tablet 08042cc961ff4aae99fd31c466823296 (DEFAULT_TABLE table=test-table [id=eee78824156448ffb2cf43fe2309a2f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:34:56.356647 27622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08042cc961ff4aae99fd31c466823296. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:34:56.366371 28164 tablet_bootstrap.cc:492] T 08042cc961ff4aae99fd31c466823296 P 5afe3e6f759e42e2b64a820f20656205: Bootstrap starting.
I20260811 06:34:56.369618 28165 tablet_bootstrap.cc:492] T 08042cc961ff4aae99fd31c466823296 P ce9c13504c3d4110b12f825ee9654b08: Bootstrap starting.
I20260811 06:34:56.371372 28164 tablet_bootstrap.cc:654] T 08042cc961ff4aae99fd31c466823296 P 5afe3e6f759e42e2b64a820f20656205: Neither blocks nor log segments found. Creating new log.
I20260811 06:34:56.371558 28166 tablet_bootstrap.cc:492] T 08042cc961ff4aae99fd31c466823296 P 38dbe8736e09473db77120b314f24838: Bootstrap starting.
I20260811 06:34:56.372879 28164 log.cc:824] T 08042cc961ff4aae99fd31c466823296 P 5afe3e6f759e42e2b64a820f20656205: Log is configured to *not* fsync() on all Append() calls
I20260811 06:34:56.374109 28165 tablet_bootstrap.cc:654] T 08042cc961ff4aae99fd31c466823296 P ce9c13504c3d4110b12f825ee9654b08: Neither blocks nor log segments found. Creating new log.
I20260811 06:34:56.375438 28166 tablet_bootstrap.cc:654] T 08042cc961ff4aae99fd31c466823296 P 38dbe8736e09473db77120b314f24838: Neither blocks nor log segments found. Creating new log.
I20260811 06:34:56.375502 28165 log.cc:824] T 08042cc961ff4aae99fd31c466823296 P ce9c13504c3d4110b12f825ee9654b08: Log is configured to *not* fsync() on all Append() calls
I20260811 06:34:56.376706 28166 log.cc:824] T 08042cc961ff4aae99fd31c466823296 P 38dbe8736e09473db77120b314f24838: Log is configured to *not* fsync() on all Append() calls
I20260811 06:34:56.376895 28164 tablet_bootstrap.cc:492] T 08042cc961ff4aae99fd31c466823296 P 5afe3e6f759e42e2b64a820f20656205: No bootstrap required, opened a new log
I20260811 06:34:56.377353 28164 ts_tablet_manager.cc:1397] T 08042cc961ff4aae99fd31c466823296 P 5afe3e6f759e42e2b64a820f20656205: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20260811 06:34:56.379880 28165 tablet_bootstrap.cc:492] T 08042cc961ff4aae99fd31c466823296 P ce9c13504c3d4110b12f825ee9654b08: No bootstrap required, opened a new log
I20260811 06:34:56.380177 28165 ts_tablet_manager.cc:1397] T 08042cc961ff4aae99fd31c466823296 P ce9c13504c3d4110b12f825ee9654b08: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.004s
I20260811 06:34:56.385715 28166 tablet_bootstrap.cc:492] T 08042cc961ff4aae99fd31c466823296 P 38dbe8736e09473db77120b314f24838: No bootstrap required, opened a new log
I20260811 06:34:56.385998 28166 ts_tablet_manager.cc:1397] T 08042cc961ff4aae99fd31c466823296 P 38dbe8736e09473db77120b314f24838: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.013s
I20260811 06:34:56.389420 28164 raft_consensus.cc:348] T 08042cc961ff4aae99fd31c466823296 P 5afe3e6f759e42e2b64a820f20656205 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5afe3e6f759e42e2b64a820f20656205" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 34521 } } peers { permanent_uuid: "ce9c13504c3d4110b12f825ee9654b08" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37571 } } peers { permanent_uuid: "38dbe8736e09473db77120b314f24838" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 34063 } }
I20260811 06:34:56.389968 28164 raft_consensus.cc:374] T 08042cc961ff4aae99fd31c466823296 P 5afe3e6f759e42e2b64a820f20656205 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:34:56.390084 28164 raft_consensus.cc:729] T 08042cc961ff4aae99fd31c466823296 P 5afe3e6f759e42e2b64a820f20656205 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5afe3e6f759e42e2b64a820f20656205, State: Initialized, Role: FOLLOWER
I20260811 06:34:56.390908 28164 consensus_queue.cc:260] T 08042cc961ff4aae99fd31c466823296 P 5afe3e6f759e42e2b64a820f20656205 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5afe3e6f759e42e2b64a820f20656205" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 34521 } } peers { permanent_uuid: "ce9c13504c3d4110b12f825ee9654b08" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37571 } } peers { permanent_uuid: "38dbe8736e09473db77120b314f24838" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 34063 } }
I20260811 06:34:56.391619 28165 raft_consensus.cc:348] T 08042cc961ff4aae99fd31c466823296 P ce9c13504c3d4110b12f825ee9654b08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5afe3e6f759e42e2b64a820f20656205" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 34521 } } peers { permanent_uuid: "ce9c13504c3d4110b12f825ee9654b08" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37571 } } peers { permanent_uuid: "38dbe8736e09473db77120b314f24838" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 34063 } }
I20260811 06:34:56.392205 28165 raft_consensus.cc:374] T 08042cc961ff4aae99fd31c466823296 P ce9c13504c3d4110b12f825ee9654b08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:34:56.392330 28165 raft_consensus.cc:729] T 08042cc961ff4aae99fd31c466823296 P ce9c13504c3d4110b12f825ee9654b08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce9c13504c3d4110b12f825ee9654b08, State: Initialized, Role: FOLLOWER
I20260811 06:34:56.393380 28165 consensus_queue.cc:260] T 08042cc961ff4aae99fd31c466823296 P ce9c13504c3d4110b12f825ee9654b08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5afe3e6f759e42e2b64a820f20656205" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 34521 } } peers { permanent_uuid: "ce9c13504c3d4110b12f825ee9654b08" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37571 } } peers { permanent_uuid: "38dbe8736e09473db77120b314f24838" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 34063 } }
I20260811 06:34:56.396409 28166 raft_consensus.cc:348] T 08042cc961ff4aae99fd31c466823296 P 38dbe8736e09473db77120b314f24838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5afe3e6f759e42e2b64a820f20656205" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 34521 } } peers { permanent_uuid: "ce9c13504c3d4110b12f825ee9654b08" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37571 } } peers { permanent_uuid: "38dbe8736e09473db77120b314f24838" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 34063 } }
I20260811 06:34:56.396821 28166 raft_consensus.cc:374] T 08042cc961ff4aae99fd31c466823296 P 38dbe8736e09473db77120b314f24838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:34:56.396925 28166 raft_consensus.cc:729] T 08042cc961ff4aae99fd31c466823296 P 38dbe8736e09473db77120b314f24838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38dbe8736e09473db77120b314f24838, State: Initialized, Role: FOLLOWER
I20260811 06:34:56.397594 28166 consensus_queue.cc:260] T 08042cc961ff4aae99fd31c466823296 P 38dbe8736e09473db77120b314f24838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5afe3e6f759e42e2b64a820f20656205" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 34521 } } peers { permanent_uuid: "ce9c13504c3d4110b12f825ee9654b08" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37571 } } peers { permanent_uuid: "38dbe8736e09473db77120b314f24838" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 34063 } }
I20260811 06:34:56.398818 28133 heartbeater.cc:503] Master 127.28.239.254:38423 was elected leader, sending a full tablet report...
I20260811 06:34:56.400012 28164 ts_tablet_manager.cc:1428] T 08042cc961ff4aae99fd31c466823296 P 5afe3e6f759e42e2b64a820f20656205: Time spent starting tablet: real 0.022s user 0.017s sys 0.004s
I20260811 06:34:56.401160 27921 heartbeater.cc:503] Master 127.28.239.254:38423 was elected leader, sending a full tablet report...
I20260811 06:34:56.401990 28165 ts_tablet_manager.cc:1428] T 08042cc961ff4aae99fd31c466823296 P ce9c13504c3d4110b12f825ee9654b08: Time spent starting tablet: real 0.022s user 0.019s sys 0.005s
I20260811 06:34:56.407845 28166 ts_tablet_manager.cc:1428] T 08042cc961ff4aae99fd31c466823296 P 38dbe8736e09473db77120b314f24838: Time spent starting tablet: real 0.022s user 0.012s sys 0.003s
W20260811 06:34:56.408807 27922 tablet.cc:2369] T 08042cc961ff4aae99fd31c466823296 P ce9c13504c3d4110b12f825ee9654b08: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 06:34:56.503645 28135 tablet.cc:2369] T 08042cc961ff4aae99fd31c466823296 P 5afe3e6f759e42e2b64a820f20656205: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 06:34:56.564538 27719 tablet.cc:2369] T 08042cc961ff4aae99fd31c466823296 P 38dbe8736e09473db77120b314f24838: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:34:56.617822 28171 raft_consensus.cc:482] T 08042cc961ff4aae99fd31c466823296 P 5afe3e6f759e42e2b64a820f20656205 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:34:56.618149 28171 raft_consensus.cc:504] T 08042cc961ff4aae99fd31c466823296 P 5afe3e6f759e42e2b64a820f20656205 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5afe3e6f759e42e2b64a820f20656205" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 34521 } } peers { permanent_uuid: "ce9c13504c3d4110b12f825ee9654b08" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37571 } } peers { permanent_uuid: "38dbe8736e09473db77120b314f24838" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 34063 } }
I20260811 06:34:56.619812 28171 leader_election.cc:288] T 08042cc961ff4aae99fd31c466823296 P 5afe3e6f759e42e2b64a820f20656205 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce9c13504c3d4110b12f825ee9654b08 (127.28.239.194:37571), 38dbe8736e09473db77120b314f24838 (127.28.239.193:34063)
I20260811 06:34:56.623068 28174 raft_consensus.cc:482] T 08042cc961ff4aae99fd31c466823296 P ce9c13504c3d4110b12f825ee9654b08 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:34:56.623407 28174 raft_consensus.cc:504] T 08042cc961ff4aae99fd31c466823296 P ce9c13504c3d4110b12f825ee9654b08 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5afe3e6f759e42e2b64a820f20656205" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 34521 } } peers { permanent_uuid: "ce9c13504c3d4110b12f825ee9654b08" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37571 } } peers { permanent_uuid: "38dbe8736e09473db77120b314f24838" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 34063 } }
I20260811 06:34:56.625602 28174 leader_election.cc:288] T 08042cc961ff4aae99fd31c466823296 P ce9c13504c3d4110b12f825ee9654b08 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5afe3e6f759e42e2b64a820f20656205 (127.28.239.195:34521), 38dbe8736e09473db77120b314f24838 (127.28.239.193:34063)
I20260811 06:34:56.627465 28138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:56.620311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.194:37571 (local address 127.28.239.195:47745)
0811 06:34:56.620735 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:56.620758 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 06:34:56.620939 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:34:56.621357 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:34:56.621363 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:34:56.621392 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:34:56.621671 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:56.621688 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:56.623029 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:56.623035 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:56.624613 (+ 1578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:56.624627 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:56.626377 (+ 1750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:56.626387 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:56.626661 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:56.626717 (+ 56us) client_negotiation.cc:769] Sending connection context
0811 06:34:56.626827 (+ 110us) client_negotiation.cc:240] Negotiation successful
0811 06:34:56.627020 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246}
I20260811 06:34:56.628423 28159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:56.620418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.195:47745 (local address 127.28.239.194:37571)
0811 06:34:56.620682 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0811 06:34:56.620690 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:34:56.620867 (+ 177us) server_negotiation.cc:407] Connection header received
0811 06:34:56.621125 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:34:56.621129 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:34:56.621191 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:34:56.621312 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:34:56.621848 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:56.622853 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:56.624798 (+ 1945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:56.625898 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:56.626054 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:56.626196 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0811 06:34:56.627830 (+ 1634us) server_negotiation.cc:299] Negotiation successful
0811 06:34:56.628036 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260811 06:34:56.628736 28163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:56.620320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.195:40207 (local address 127.28.239.193:34063)
0811 06:34:56.620535 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 06:34:56.620541 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:34:56.621286 (+ 745us) server_negotiation.cc:407] Connection header received
0811 06:34:56.621446 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:34:56.621453 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:34:56.621590 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:34:56.621695 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:34:56.622306 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:56.623366 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:56.625877 (+ 2511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:56.627323 (+ 1446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:56.627461 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:56.627536 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 06:34:56.628211 (+ 675us) server_negotiation.cc:299] Negotiation successful
0811 06:34:56.628397 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 06:34:56.629756 28180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:56.620356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.193:34063 (local address 127.28.239.195:40207)
0811 06:34:56.621166 (+ 810us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:56.621180 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 06:34:56.621308 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:34:56.621757 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:34:56.621763 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:34:56.621812 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:34:56.622135 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:56.622143 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:56.623563 (+ 1420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:56.623572 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:56.625647 (+ 2075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:56.625663 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:56.627722 (+ 2059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:56.627730 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:56.627988 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:56.628038 (+ 50us) client_negotiation.cc:769] Sending connection context
0811 06:34:56.629071 (+ 1033us) client_negotiation.cc:240] Negotiation successful
0811 06:34:56.629257 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":707,"thread_start_us":449,"threads_started":1}
I20260811 06:34:56.630131 27858 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08042cc961ff4aae99fd31c466823296" candidate_uuid: "5afe3e6f759e42e2b64a820f20656205" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce9c13504c3d4110b12f825ee9654b08" is_pre_election: true
I20260811 06:34:56.631426 27650 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08042cc961ff4aae99fd31c466823296" candidate_uuid: "5afe3e6f759e42e2b64a820f20656205" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38dbe8736e09473db77120b314f24838" is_pre_election: true
I20260811 06:34:56.631899 27858 raft_consensus.cc:2454] T 08042cc961ff4aae99fd31c466823296 P ce9c13504c3d4110b12f825ee9654b08 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5afe3e6f759e42e2b64a820f20656205 in term 0.
I20260811 06:34:56.631963 27650 raft_consensus.cc:2454] T 08042cc961ff4aae99fd31c466823296 P 38dbe8736e09473db77120b314f24838 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5afe3e6f759e42e2b64a820f20656205 in term 0.
I20260811 06:34:56.632781 27955 leader_election.cc:302] T 08042cc961ff4aae99fd31c466823296 P 5afe3e6f759e42e2b64a820f20656205 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38dbe8736e09473db77120b314f24838, 5afe3e6f759e42e2b64a820f20656205; no voters:
I20260811 06:34:56.633594 28171 raft_consensus.cc:2781] T 08042cc961ff4aae99fd31c466823296 P 5afe3e6f759e42e2b64a820f20656205 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:34:56.633760 28171 raft_consensus.cc:482] T 08042cc961ff4aae99fd31c466823296 P 5afe3e6f759e42e2b64a820f20656205 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:34:56.633898 28171 raft_consensus.cc:3037] T 08042cc961ff4aae99fd31c466823296 P 5afe3e6f759e42e2b64a820f20656205 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:34:56.637353 28171 raft_consensus.cc:504] T 08042cc961ff4aae99fd31c466823296 P 5afe3e6f759e42e2b64a820f20656205 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5afe3e6f759e42e2b64a820f20656205" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 34521 } } peers { permanent_uuid: "ce9c13504c3d4110b12f825ee9654b08" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37571 } } peers { permanent_uuid: "38dbe8736e09473db77120b314f24838" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 34063 } }
I20260811 06:34:56.638619 27858 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08042cc961ff4aae99fd31c466823296" candidate_uuid: "5afe3e6f759e42e2b64a820f20656205" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce9c13504c3d4110b12f825ee9654b08"
I20260811 06:34:56.638744 27650 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08042cc961ff4aae99fd31c466823296" candidate_uuid: "5afe3e6f759e42e2b64a820f20656205" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38dbe8736e09473db77120b314f24838"
I20260811 06:34:56.638859 27858 raft_consensus.cc:3037] T 08042cc961ff4aae99fd31c466823296 P ce9c13504c3d4110b12f825ee9654b08 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:34:56.638974 27650 raft_consensus.cc:3037] T 08042cc961ff4aae99fd31c466823296 P 38dbe8736e09473db77120b314f24838 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:34:56.639725 28153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:56.627395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.194:43209 (local address 127.28.239.195:34521)
0811 06:34:56.627619 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0811 06:34:56.627627 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:34:56.632010 (+ 4383us) server_negotiation.cc:407] Connection header received
0811 06:34:56.632297 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:34:56.632303 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:34:56.632381 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:34:56.632446 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:34:56.633297 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:56.634568 (+ 1271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:56.637169 (+ 2601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:56.638483 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:56.638658 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:56.638727 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 06:34:56.639340 (+ 613us) server_negotiation.cc:299] Negotiation successful
0811 06:34:56.639478 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 06:34:56.640566 28184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:56.625824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.195:34521 (local address 127.28.239.194:43209)
0811 06:34:56.631904 (+ 6080us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:56.631926 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 06:34:56.632134 (+ 208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:34:56.632654 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:34:56.632658 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:34:56.632686 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:34:56.633118 (+ 432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:56.633132 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:56.634765 (+ 1633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:56.634772 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:56.636978 (+ 2206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:56.636996 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:56.638911 (+ 1915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:56.638918 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:56.639106 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:56.639157 (+ 51us) client_negotiation.cc:769] Sending connection context
0811 06:34:56.639951 (+ 794us) client_negotiation.cc:240] Negotiation successful
0811 06:34:56.640160 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4648,"thread_start_us":7344,"threads_started":1}
I20260811 06:34:56.640673 28163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:56.626724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.194:38949 (local address 127.28.239.193:34063)
0811 06:34:56.629017 (+ 2293us) server_negotiation.cc:206] Beginning negotiation
0811 06:34:56.629026 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:34:56.633562 (+ 4536us) server_negotiation.cc:407] Connection header received
0811 06:34:56.633877 (+ 315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:34:56.633884 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:34:56.633961 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:34:56.634053 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:34:56.634653 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:56.635705 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:56.637746 (+ 2041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:56.639055 (+ 1309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:56.639249 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:56.639340 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 06:34:56.640219 (+ 879us) server_negotiation.cc:299] Negotiation successful
0811 06:34:56.640368 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2165,"thread_start_us":3803,"threads_started":1}
I20260811 06:34:56.641654 28181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:56.626268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.193:34063 (local address 127.28.239.194:38949)
0811 06:34:56.633456 (+ 7188us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:56.633471 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 06:34:56.633634 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:34:56.634067 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:34:56.634072 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:34:56.634107 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:34:56.634417 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:56.634450 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:56.635888 (+ 1438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:56.635892 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:56.637494 (+ 1602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:56.637509 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:56.639585 (+ 2076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:56.639594 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:56.639948 (+ 354us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:56.639984 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 06:34:56.641068 (+ 1084us) client_negotiation.cc:240] Negotiation successful
0811 06:34:56.641238 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7071,"thread_start_us":4268,"threads_started":1}
I20260811 06:34:56.641736 28171 leader_election.cc:288] T 08042cc961ff4aae99fd31c466823296 P 5afe3e6f759e42e2b64a820f20656205 [CANDIDATE]: Term 1 election: Requested vote from peers ce9c13504c3d4110b12f825ee9654b08 (127.28.239.194:37571), 38dbe8736e09473db77120b314f24838 (127.28.239.193:34063)
I20260811 06:34:56.642385 27858 raft_consensus.cc:2454] T 08042cc961ff4aae99fd31c466823296 P ce9c13504c3d4110b12f825ee9654b08 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5afe3e6f759e42e2b64a820f20656205 in term 1.
I20260811 06:34:56.642488 27650 raft_consensus.cc:2454] T 08042cc961ff4aae99fd31c466823296 P 38dbe8736e09473db77120b314f24838 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5afe3e6f759e42e2b64a820f20656205 in term 1.
I20260811 06:34:56.642602 27648 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08042cc961ff4aae99fd31c466823296" candidate_uuid: "ce9c13504c3d4110b12f825ee9654b08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38dbe8736e09473db77120b314f24838" is_pre_election: true
I20260811 06:34:56.642962 27955 leader_election.cc:302] T 08042cc961ff4aae99fd31c466823296 P 5afe3e6f759e42e2b64a820f20656205 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38dbe8736e09473db77120b314f24838, 5afe3e6f759e42e2b64a820f20656205; no voters:
I20260811 06:34:56.643034 27648 raft_consensus.cc:2379] T 08042cc961ff4aae99fd31c466823296 P 38dbe8736e09473db77120b314f24838 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce9c13504c3d4110b12f825ee9654b08 in current term 1: Already voted for candidate 5afe3e6f759e42e2b64a820f20656205 in this term.
I20260811 06:34:56.643285 28171 raft_consensus.cc:2781] T 08042cc961ff4aae99fd31c466823296 P 5afe3e6f759e42e2b64a820f20656205 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:34:56.643122 28052 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08042cc961ff4aae99fd31c466823296" candidate_uuid: "ce9c13504c3d4110b12f825ee9654b08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5afe3e6f759e42e2b64a820f20656205" is_pre_election: true
I20260811 06:34:56.643999 28171 raft_consensus.cc:686] T 08042cc961ff4aae99fd31c466823296 P 5afe3e6f759e42e2b64a820f20656205 [term 1 LEADER]: Becoming Leader. State: Replica: 5afe3e6f759e42e2b64a820f20656205, State: Running, Role: LEADER
I20260811 06:34:56.644474 28171 consensus_queue.cc:237] T 08042cc961ff4aae99fd31c466823296 P 5afe3e6f759e42e2b64a820f20656205 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5afe3e6f759e42e2b64a820f20656205" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 34521 } } peers { permanent_uuid: "ce9c13504c3d4110b12f825ee9654b08" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37571 } } peers { permanent_uuid: "38dbe8736e09473db77120b314f24838" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 34063 } }
I20260811 06:34:56.648952 27758 leader_election.cc:302] T 08042cc961ff4aae99fd31c466823296 P ce9c13504c3d4110b12f825ee9654b08 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce9c13504c3d4110b12f825ee9654b08; no voters: 38dbe8736e09473db77120b314f24838, 5afe3e6f759e42e2b64a820f20656205
I20260811 06:34:56.649367 28174 raft_consensus.cc:2726] T 08042cc961ff4aae99fd31c466823296 P ce9c13504c3d4110b12f825ee9654b08 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 06:34:56.650135 27443 catalog_manager.cc:5495] T 08042cc961ff4aae99fd31c466823296 P 5afe3e6f759e42e2b64a820f20656205 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5afe3e6f759e42e2b64a820f20656205 (127.28.239.195). New cstate: current_term: 1 leader_uuid: "5afe3e6f759e42e2b64a820f20656205" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5afe3e6f759e42e2b64a820f20656205" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 34521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce9c13504c3d4110b12f825ee9654b08" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38dbe8736e09473db77120b314f24838" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 34063 } health_report { overall_health: UNKNOWN } } }
I20260811 06:34:56.720203 28153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:56.714745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42868 (local address 127.28.239.195:34521)
0811 06:34:56.714926 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 06:34:56.714931 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:34:56.714949 (+ 18us) server_negotiation.cc:407] Connection header received
0811 06:34:56.715045 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:34:56.715049 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:34:56.715196 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:34:56.715341 (+ 145us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:34:56.715710 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:56.716611 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:56.717987 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:56.718452 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:56.718558 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:56.719015 (+ 457us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:34:56.719611 (+ 596us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:34:56.719734 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 06:34:56.719841 (+ 107us) server_negotiation.cc:299] Negotiation successful
0811 06:34:56.719913 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 06:34:56.729383 27858 raft_consensus.cc:1261] T 08042cc961ff4aae99fd31c466823296 P ce9c13504c3d4110b12f825ee9654b08 [term 1 FOLLOWER]: Refusing update from remote peer 5afe3e6f759e42e2b64a820f20656205: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:34:56.729367 27648 raft_consensus.cc:1261] T 08042cc961ff4aae99fd31c466823296 P 38dbe8736e09473db77120b314f24838 [term 1 FOLLOWER]: Refusing update from remote peer 5afe3e6f759e42e2b64a820f20656205: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:34:56.730484 28187 consensus_queue.cc:1035] T 08042cc961ff4aae99fd31c466823296 P 5afe3e6f759e42e2b64a820f20656205 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce9c13504c3d4110b12f825ee9654b08" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 37571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:34:56.730903 28171 consensus_queue.cc:1035] T 08042cc961ff4aae99fd31c466823296 P 5afe3e6f759e42e2b64a820f20656205 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38dbe8736e09473db77120b314f24838" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 34063 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:34:56.757057 27443 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36586:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:34:56.757406 27443 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36586:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:34:56.760953 27443 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8e691a02f86a47388427deb0843c8967: Sending DeleteTablet for 3 replicas of tablet 08042cc961ff4aae99fd31c466823296
I20260811 06:34:56.762425 28024 tablet_service.cc:1514] Processing DeleteTablet for tablet 08042cc961ff4aae99fd31c466823296 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:34:56 UTC) from {username='kudu', principal='kudu/127.28.239.254@KRBTEST.COM'} at 127.0.0.1:42856
I20260811 06:34:56.762480 27820 tablet_service.cc:1514] Processing DeleteTablet for tablet 08042cc961ff4aae99fd31c466823296 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:34:56 UTC) from {username='kudu', principal='kudu/127.28.239.254@KRBTEST.COM'} at 127.0.0.1:54584
I20260811 06:34:56.762971 27622 tablet_service.cc:1514] Processing DeleteTablet for tablet 08042cc961ff4aae99fd31c466823296 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:34:56 UTC) from {username='kudu', principal='kudu/127.28.239.254@KRBTEST.COM'} at 127.0.0.1:44630
I20260811 06:34:56.766916 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 27535
I20260811 06:34:56.778045 28208 tablet_replica.cc:330] T 08042cc961ff4aae99fd31c466823296 P 5afe3e6f759e42e2b64a820f20656205: stopping tablet replica
I20260811 06:34:56.778709 28208 raft_consensus.cc:2229] T 08042cc961ff4aae99fd31c466823296 P 5afe3e6f759e42e2b64a820f20656205 [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:34:56.779588 28208 raft_consensus.cc:2258] T 08042cc961ff4aae99fd31c466823296 P 5afe3e6f759e42e2b64a820f20656205 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:34:56.780459 28209 tablet_replica.cc:330] T 08042cc961ff4aae99fd31c466823296 P ce9c13504c3d4110b12f825ee9654b08: stopping tablet replica
I20260811 06:34:56.780859 28209 raft_consensus.cc:2229] T 08042cc961ff4aae99fd31c466823296 P ce9c13504c3d4110b12f825ee9654b08 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:34:56.781324 28209 raft_consensus.cc:2258] T 08042cc961ff4aae99fd31c466823296 P ce9c13504c3d4110b12f825ee9654b08 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:34:56.783142 28208 ts_tablet_manager.cc:1905] T 08042cc961ff4aae99fd31c466823296 P 5afe3e6f759e42e2b64a820f20656205: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:34:56.786684 28210 tablet_replica.cc:330] T 08042cc961ff4aae99fd31c466823296 P 38dbe8736e09473db77120b314f24838: stopping tablet replica
I20260811 06:34:56.787173 28210 raft_consensus.cc:2229] T 08042cc961ff4aae99fd31c466823296 P 38dbe8736e09473db77120b314f24838 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:34:56.787669 28209 ts_tablet_manager.cc:1905] T 08042cc961ff4aae99fd31c466823296 P ce9c13504c3d4110b12f825ee9654b08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:34:56.789160 28210 raft_consensus.cc:2258] T 08042cc961ff4aae99fd31c466823296 P 38dbe8736e09473db77120b314f24838 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:34:56.791059 28208 ts_tablet_manager.cc:1918] T 08042cc961ff4aae99fd31c466823296 P 5afe3e6f759e42e2b64a820f20656205: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:34:56.791273 28208 log.cc:1196] T 08042cc961ff4aae99fd31c466823296 P 5afe3e6f759e42e2b64a820f20656205: Deleting WAL directory at /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/minicluster-data/ts-2/wal/wals/08042cc961ff4aae99fd31c466823296
I20260811 06:34:56.791929 28208 ts_tablet_manager.cc:1939] T 08042cc961ff4aae99fd31c466823296 P 5afe3e6f759e42e2b64a820f20656205: Deleting consensus metadata
I20260811 06:34:56.794085 27426 catalog_manager.cc:4841] TS 5afe3e6f759e42e2b64a820f20656205 (127.28.239.195:34521): tablet 08042cc961ff4aae99fd31c466823296 (table test-table [id=eee78824156448ffb2cf43fe2309a2f3]) successfully deleted
I20260811 06:34:56.794872 28210 ts_tablet_manager.cc:1905] T 08042cc961ff4aae99fd31c466823296 P 38dbe8736e09473db77120b314f24838: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:34:56.795003 28209 ts_tablet_manager.cc:1918] T 08042cc961ff4aae99fd31c466823296 P ce9c13504c3d4110b12f825ee9654b08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:34:56.795195 28209 log.cc:1196] T 08042cc961ff4aae99fd31c466823296 P ce9c13504c3d4110b12f825ee9654b08: Deleting WAL directory at /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/minicluster-data/ts-1/wal/wals/08042cc961ff4aae99fd31c466823296
I20260811 06:34:56.795748 28209 ts_tablet_manager.cc:1939] T 08042cc961ff4aae99fd31c466823296 P ce9c13504c3d4110b12f825ee9654b08: Deleting consensus metadata
I20260811 06:34:56.797640 27424 catalog_manager.cc:4841] TS ce9c13504c3d4110b12f825ee9654b08 (127.28.239.194:37571): tablet 08042cc961ff4aae99fd31c466823296 (table test-table [id=eee78824156448ffb2cf43fe2309a2f3]) successfully deleted
I20260811 06:34:56.799978 28210 ts_tablet_manager.cc:1918] T 08042cc961ff4aae99fd31c466823296 P 38dbe8736e09473db77120b314f24838: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:34:56.800163 28210 log.cc:1196] T 08042cc961ff4aae99fd31c466823296 P 38dbe8736e09473db77120b314f24838: Deleting WAL directory at /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786429755927508-29631-0/minicluster-data/ts-0/wal/wals/08042cc961ff4aae99fd31c466823296
I20260811 06:34:56.800742 28210 ts_tablet_manager.cc:1939] T 08042cc961ff4aae99fd31c466823296 P 38dbe8736e09473db77120b314f24838: Deleting consensus metadata
I20260811 06:34:56.802660 27427 catalog_manager.cc:4841] TS 38dbe8736e09473db77120b314f24838 (127.28.239.193:34063): tablet 08042cc961ff4aae99fd31c466823296 (table test-table [id=eee78824156448ffb2cf43fe2309a2f3]) successfully deleted
Aug 11 06:34:56 dist-test-slave-gkw7 krb5kdc[27379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430093, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.239.193@KRBTEST.COM
I20260811 06:34:56.811179 28163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:56.768392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44642 (local address 127.28.239.193:34063)
0811 06:34:56.768583 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 06:34:56.768593 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:34:56.783868 (+ 15275us) server_negotiation.cc:407] Connection header received
0811 06:34:56.784186 (+ 318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:34:56.784194 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:34:56.784301 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:34:56.784375 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:34:56.787432 (+ 3057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:56.788535 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:56.794584 (+ 6049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:56.795303 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:56.795454 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:56.805152 (+ 9698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:34:56.805165 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:34:56.805180 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:34:56.805226 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:34:56.808245 (+ 3019us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:34:56.809005 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:34:56.809013 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:34:56.809015 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:34:56.809150 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:34:56.809587 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:34:56.809592 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:34:56.809599 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:34:56.809959 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:34:56.810283 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0811 06:34:56.810617 (+ 334us) server_negotiation.cc:299] Negotiation successful
0811 06:34:56.810854 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 06:34:56.811862 27711 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:34:56.905014 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 27535
I20260811 06:34:56.934520 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 27724
Aug 11 06:34:56 dist-test-slave-gkw7 krb5kdc[27379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430093, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.239.194@KRBTEST.COM
I20260811 06:34:56.951473 28159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:56.935455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54588 (local address 127.28.239.194:37571)
0811 06:34:56.935627 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 06:34:56.935634 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:34:56.935712 (+ 78us) server_negotiation.cc:407] Connection header received
0811 06:34:56.935940 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:34:56.935945 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:34:56.936027 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:34:56.936134 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:34:56.937489 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:56.938325 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:56.939606 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:56.940070 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:56.940215 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:56.945245 (+ 5030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:34:56.945262 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:34:56.945279 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:34:56.945319 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:34:56.948433 (+ 3114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:34:56.949184 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:34:56.949192 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:34:56.949195 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:34:56.949333 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:34:56.949810 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:34:56.949815 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:34:56.949820 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:34:56.950223 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:34:56.950550 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0811 06:34:56.950871 (+ 321us) server_negotiation.cc:299] Negotiation successful
0811 06:34:56.951104 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 06:34:56.952147 27914 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:34:57.046651 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 27724
I20260811 06:34:57.083638 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 27929
Aug 11 06:34:57 dist-test-slave-gkw7 krb5kdc[27379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430093, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.239.195@KRBTEST.COM
I20260811 06:34:57.100804 28153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:57.084465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42872 (local address 127.28.239.195:34521)
0811 06:34:57.084768 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0811 06:34:57.084774 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:34:57.084796 (+ 22us) server_negotiation.cc:407] Connection header received
0811 06:34:57.084914 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:34:57.084921 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:34:57.084991 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:34:57.085115 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:34:57.086686 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:57.087508 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:57.088829 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:57.089310 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:57.089449 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:57.094308 (+ 4859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:34:57.094323 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:34:57.094340 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:34:57.094380 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:34:57.097811 (+ 3431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:34:57.098581 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:34:57.098592 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:34:57.098601 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:34:57.098733 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:34:57.099151 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:34:57.099155 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:34:57.099157 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:34:57.099554 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:34:57.099933 (+ 379us) server_negotiation.cc:1035] Waiting for connection context
0811 06:34:57.100220 (+ 287us) server_negotiation.cc:299] Negotiation successful
0811 06:34:57.100471 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123}
I20260811 06:34:57.101756 28128 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:34:57.230479 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 27929
I20260811 06:34:57.262790 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 27399
I20260811 06:34:57.263928 27488 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:34:57.381857 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 27399
2026-08-11T06:34:57Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (5151 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-11T06:34:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:34:57Z Disabled control of system clock
I20260811 06:34:57.487746 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429755927508-29631-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429755927508-29631-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429755927508-29631-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:37995
--webserver_interface=127.28.239.254
--webserver_port=0
--builtin_ntp_servers=127.28.239.212:40573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.239.254:37995
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 06:34:57.858516 28222 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:34:57.858809 28222 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:34:57.858893 28222 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:34:57.866070 28222 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:34:57.866183 28222 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:34:57.866230 28222 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:34:57.866289 28222 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:34:57.866328 28222 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:34:57.875478 28222 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:40573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429755927508-29631-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429755927508-29631-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.239.254:37995
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:37995
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.239.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429755927508-29631-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:34:57.876780 28222 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:34:57.878636 28222 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:34:57.890868 28230 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:34:57.890961 28228 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:34:57.890934 28232 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:34:57.892118 28222 server_base.cc:1034] running on GCE node
I20260811 06:34:57.892942 28222 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:34:57.894492 28222 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:34:57.895737 28222 hybrid_clock.cc:648] HybridClock initialized: now 1786430097895691 us; error 56 us; skew 500 ppm
I20260811 06:34:57.896136 28222 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:34:57.898593 28222 webserver.cc:460] Webserver started at http://127.28.239.254:34477/ using document root <none> and password file <none>
I20260811 06:34:57.899356 28222 fs_manager.cc:359] Metadata directory not provided
I20260811 06:34:57.899498 28222 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:34:57.899850 28222 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:34:57.902989 28222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429755927508-29631-0/minicluster-data/master-0/data/instance:
uuid: "fa9d87218ab8425aa52f1b08aea67667"
format_stamp: "Formatted at 2026-08-11 06:34:57 on dist-test-slave-gkw7"
server_key: "0ca144638b37b604f4d50123310020e0"
server_key_iv: "cf4219da26fd6f9c5d88961ddd388e28"
server_key_version: "encryptionkey@0"
I20260811 06:34:57.903856 28222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429755927508-29631-0/minicluster-data/master-0/wal/instance:
uuid: "fa9d87218ab8425aa52f1b08aea67667"
format_stamp: "Formatted at 2026-08-11 06:34:57 on dist-test-slave-gkw7"
server_key: "0ca144638b37b604f4d50123310020e0"
server_key_iv: "cf4219da26fd6f9c5d88961ddd388e28"
server_key_version: "encryptionkey@0"
I20260811 06:34:57.909206 28222 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 06:34:57.912904 28240 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:34:57.914324 28222 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 06:34:57.914523 28222 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429755927508-29631-0/minicluster-data/master-0/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429755927508-29631-0/minicluster-data/master-0/wal
uuid: "fa9d87218ab8425aa52f1b08aea67667"
format_stamp: "Formatted at 2026-08-11 06:34:57 on dist-test-slave-gkw7"
server_key: "0ca144638b37b604f4d50123310020e0"
server_key_iv: "cf4219da26fd6f9c5d88961ddd388e28"
server_key_version: "encryptionkey@0"
I20260811 06:34:57.914763 28222 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429755927508-29631-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429755927508-29631-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429755927508-29631-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:34:57.949684 28222 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:34:57.950642 28222 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:34:57.950969 28222 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:34:57.970791 28222 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.254:37995
I20260811 06:34:57.970804 28319 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.254:37995 every 8 connection(s)
I20260811 06:34:57.972246 28222 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
I20260811 06:34:57.974529 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 28222
I20260811 06:34:57.974768 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429755927508-29631-0/minicluster-data/master-0/wal/instance
I20260811 06:34:57.975237 29631 external_mini_cluster.cc:1428] Setting key 268b6e49a11d9c2edeff2b091b2a0aca
I20260811 06:34:57.976795 28320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:34:57.989967 28323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:57.977390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56714 (local address 127.28.239.254:37995)
0811 06:34:57.978500 (+ 1110us) server_negotiation.cc:206] Beginning negotiation
0811 06:34:57.978511 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 06:34:57.978555 (+ 44us) server_negotiation.cc:407] Connection header received
0811 06:34:57.980166 (+ 1611us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:34:57.980188 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:34:57.980685 (+ 497us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:34:57.981035 (+ 350us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:34:57.981714 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:57.983136 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:57.985084 (+ 1948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:57.985833 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:57.985968 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:57.987317 (+ 1349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:34:57.987324 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:34:57.987341 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:34:57.987966 (+ 625us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:34:57.988101 (+ 135us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:34:57.988139 (+ 38us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:34:57.988243 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 06:34:57.988484 (+ 241us) server_negotiation.cc:299] Negotiation successful
0811 06:34:57.988851 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":686,"thread_start_us":466,"threads_started":1}
I20260811 06:34:57.991331 28320 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa9d87218ab8425aa52f1b08aea67667: Bootstrap starting.
I20260811 06:34:57.995025 28320 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fa9d87218ab8425aa52f1b08aea67667: Neither blocks nor log segments found. Creating new log.
I20260811 06:34:57.996117 28320 log.cc:824] T 00000000000000000000000000000000 P fa9d87218ab8425aa52f1b08aea67667: Log is configured to *not* fsync() on all Append() calls
I20260811 06:34:57.999239 28320 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa9d87218ab8425aa52f1b08aea67667: No bootstrap required, opened a new log
I20260811 06:34:58.006837 28320 raft_consensus.cc:348] T 00000000000000000000000000000000 P fa9d87218ab8425aa52f1b08aea67667 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa9d87218ab8425aa52f1b08aea67667" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 37995 } }
I20260811 06:34:58.007210 28320 raft_consensus.cc:374] T 00000000000000000000000000000000 P fa9d87218ab8425aa52f1b08aea67667 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:34:58.007293 28320 raft_consensus.cc:729] T 00000000000000000000000000000000 P fa9d87218ab8425aa52f1b08aea67667 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa9d87218ab8425aa52f1b08aea67667, State: Initialized, Role: FOLLOWER
I20260811 06:34:58.007833 28320 consensus_queue.cc:260] T 00000000000000000000000000000000 P fa9d87218ab8425aa52f1b08aea67667 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa9d87218ab8425aa52f1b08aea67667" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 37995 } }
I20260811 06:34:58.008041 28320 raft_consensus.cc:388] T 00000000000000000000000000000000 P fa9d87218ab8425aa52f1b08aea67667 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:34:58.008134 28320 raft_consensus.cc:482] T 00000000000000000000000000000000 P fa9d87218ab8425aa52f1b08aea67667 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:34:58.008265 28320 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fa9d87218ab8425aa52f1b08aea67667 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:34:58.010823 28320 raft_consensus.cc:504] T 00000000000000000000000000000000 P fa9d87218ab8425aa52f1b08aea67667 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa9d87218ab8425aa52f1b08aea67667" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 37995 } }
I20260811 06:34:58.011309 28320 leader_election.cc:302] T 00000000000000000000000000000000 P fa9d87218ab8425aa52f1b08aea67667 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fa9d87218ab8425aa52f1b08aea67667; no voters:
I20260811 06:34:58.011859 28320 leader_election.cc:288] T 00000000000000000000000000000000 P fa9d87218ab8425aa52f1b08aea67667 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:34:58.012023 28329 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fa9d87218ab8425aa52f1b08aea67667 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:34:58.012894 28329 raft_consensus.cc:686] T 00000000000000000000000000000000 P fa9d87218ab8425aa52f1b08aea67667 [term 1 LEADER]: Becoming Leader. State: Replica: fa9d87218ab8425aa52f1b08aea67667, State: Running, Role: LEADER
I20260811 06:34:58.013433 28329 consensus_queue.cc:237] T 00000000000000000000000000000000 P fa9d87218ab8425aa52f1b08aea67667 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa9d87218ab8425aa52f1b08aea67667" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 37995 } }
I20260811 06:34:58.014223 28320 sys_catalog.cc:564] T 00000000000000000000000000000000 P fa9d87218ab8425aa52f1b08aea67667 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:34:58.016891 28333 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa9d87218ab8425aa52f1b08aea67667 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fa9d87218ab8425aa52f1b08aea67667. Latest consensus state: current_term: 1 leader_uuid: "fa9d87218ab8425aa52f1b08aea67667" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa9d87218ab8425aa52f1b08aea67667" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 37995 } } }
I20260811 06:34:58.017436 28333 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa9d87218ab8425aa52f1b08aea67667 [sys.catalog]: This master's current role is: LEADER
I20260811 06:34:58.018356 28332 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa9d87218ab8425aa52f1b08aea67667 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fa9d87218ab8425aa52f1b08aea67667" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa9d87218ab8425aa52f1b08aea67667" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 37995 } } }
I20260811 06:34:58.018743 28332 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa9d87218ab8425aa52f1b08aea67667 [sys.catalog]: This master's current role is: LEADER
I20260811 06:34:58.019501 28341 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:34:58.023877 28341 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:34:58.034307 28341 catalog_manager.cc:1347] Generated new cluster ID: 938cf4a1ff1d46f19063d11949417d1d
I20260811 06:34:58.034442 28341 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:34:58.086570 28341 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:34:58.088608 28341 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:34:58.096601 28341 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fa9d87218ab8425aa52f1b08aea67667: Generated new TSK 0
I20260811 06:34:58.097714 28341 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:34:58.108759 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429755927508-29631-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429755927508-29631-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429755927508-29631-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.193:0
--local_ip_for_outbound_sockets=127.28.239.193
--webserver_interface=127.28.239.193
--webserver_port=0
--tserver_master_addrs=127.28.239.254:37995
--builtin_ntp_servers=127.28.239.212:40573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 06:34:58.466291 28358 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:34:58.466593 28358 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:34:58.466686 28358 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:34:58.474010 28358 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:34:58.474131 28358 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:34:58.474241 28358 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.193
I20260811 06:34:58.483505 28358 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:40573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429755927508-29631-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429755927508-29631-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.239.193
--webserver_port=0
--tserver_master_addrs=127.28.239.254:37995
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.193
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429755927508-29631-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:34:58.484826 28358 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:34:58.486626 28358 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:34:58.500784 28367 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:34:58.500705 28369 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:34:58.500758 28372 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:34:58.501904 28358 server_base.cc:1034] running on GCE node
I20260811 06:34:58.502534 28358 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:34:58.503471 28358 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:34:58.504695 28358 hybrid_clock.cc:648] HybridClock initialized: now 1786430098504651 us; error 30 us; skew 500 ppm
I20260811 06:34:58.505043 28358 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:34:58.507485 28358 webserver.cc:460] Webserver started at http://127.28.239.193:38537/ using document root <none> and password file <none>
I20260811 06:34:58.508257 28358 fs_manager.cc:359] Metadata directory not provided
I20260811 06:34:58.508385 28358 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:34:58.508750 28358 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:34:58.511786 28358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429755927508-29631-0/minicluster-data/ts-0/data/instance:
uuid: "29e6eba07d584b298acefddfc15f40ba"
format_stamp: "Formatted at 2026-08-11 06:34:58 on dist-test-slave-gkw7"
server_key: "232e5c4a9e29fda10153f97444b4c447"
server_key_iv: "ebdff5313e6b7419574e7a86b24bfbdd"
server_key_version: "encryptionkey@0"
I20260811 06:34:58.512616 28358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429755927508-29631-0/minicluster-data/ts-0/wal/instance:
uuid: "29e6eba07d584b298acefddfc15f40ba"
format_stamp: "Formatted at 2026-08-11 06:34:58 on dist-test-slave-gkw7"
server_key: "232e5c4a9e29fda10153f97444b4c447"
server_key_iv: "ebdff5313e6b7419574e7a86b24bfbdd"
server_key_version: "encryptionkey@0"
I20260811 06:34:58.518360 28358 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260811 06:34:58.522600 28379 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:34:58.524077 28358 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 06:34:58.524319 28358 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429755927508-29631-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429755927508-29631-0/minicluster-data/ts-0/wal
uuid: "29e6eba07d584b298acefddfc15f40ba"
format_stamp: "Formatted at 2026-08-11 06:34:58 on dist-test-slave-gkw7"
server_key: "232e5c4a9e29fda10153f97444b4c447"
server_key_iv: "ebdff5313e6b7419574e7a86b24bfbdd"
server_key_version: "encryptionkey@0"
I20260811 06:34:58.524562 28358 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429755927508-29631-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429755927508-29631-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429755927508-29631-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:34:58.555632 28358 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:34:58.556617 28358 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:34:58.556968 28358 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:34:58.558056 28358 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:34:58.560066 28358 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:34:58.560205 28358 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:34:58.560329 28358 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:34:58.560405 28358 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:34:58.602244 28358 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.193:43215
I20260811 06:34:58.602339 28540 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.193:43215 every 8 connection(s)
I20260811 06:34:58.603621 28358 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429755927508-29631-0/minicluster-data/ts-0/data/info.pb
I20260811 06:34:58.606742 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 28358
I20260811 06:34:58.606933 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429755927508-29631-0/minicluster-data/ts-0/wal/instance
I20260811 06:34:58.607399 29631 external_mini_cluster.cc:1428] Setting key 09047660b403d78b2b79d35e6e9eee6d
I20260811 06:34:58.610724 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429755927508-29631-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429755927508-29631-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429755927508-29631-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.194:0
--local_ip_for_outbound_sockets=127.28.239.194
--webserver_interface=127.28.239.194
--webserver_port=0
--tserver_master_addrs=127.28.239.254:37995
--builtin_ntp_servers=127.28.239.212:40573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 06:34:58.619518 28544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:58.606344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.193:58633 (local address 127.28.239.254:37995)
0811 06:34:58.606882 (+ 538us) server_negotiation.cc:206] Beginning negotiation
0811 06:34:58.606890 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:34:58.608484 (+ 1594us) server_negotiation.cc:407] Connection header received
0811 06:34:58.610929 (+ 2445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:34:58.610938 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:34:58.611024 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:34:58.611128 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:34:58.612662 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:58.613384 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:58.616058 (+ 2674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:58.616507 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:58.616635 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:58.618576 (+ 1941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:34:58.618586 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:34:58.618595 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:34:58.618617 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:34:58.618671 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:34:58.618686 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:34:58.618768 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 06:34:58.619130 (+ 362us) server_negotiation.cc:299] Negotiation successful
0811 06:34:58.619224 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":251,"threads_started":1}
I20260811 06:34:58.621222 28546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:58.607153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:37995 (local address 127.28.239.193:58633)
0811 06:34:58.608255 (+ 1102us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:58.608344 (+ 89us) client_negotiation.cc:174] Beginning negotiation
0811 06:34:58.610567 (+ 2223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:34:58.611411 (+ 844us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:34:58.611425 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:34:58.611554 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:34:58.612459 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:58.612489 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:58.613591 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:58.613600 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:58.615888 (+ 2288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:58.615907 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:58.616787 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:58.616797 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:58.617209 (+ 412us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:58.618088 (+ 879us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 06:34:58.618094 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:34:58.618161 (+ 67us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 06:34:58.618278 (+ 117us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:34:58.618799 (+ 521us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:34:58.618814 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:34:58.618847 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 06:34:58.619053 (+ 206us) client_negotiation.cc:240] Negotiation successful
0811 06:34:58.619554 (+ 501us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":543,"thread_start_us":319,"threads_started":1}
I20260811 06:34:58.622970 28542 heartbeater.cc:347] Connected to a master server at 127.28.239.254:37995
I20260811 06:34:58.623332 28542 heartbeater.cc:464] Registering TS with master...
I20260811 06:34:58.624365 28542 heartbeater.cc:511] Master 127.28.239.254:37995 requested a full tablet report, sending...
I20260811 06:34:58.626940 28261 ts_manager.cc:194] Registered new tserver with Master: 29e6eba07d584b298acefddfc15f40ba (127.28.239.193:43215)
I20260811 06:34:58.628468 28261 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.28.239.193:58633
W20260811 06:34:58.972502 28547 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:34:58.972796 28547 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:34:58.972942 28547 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:34:58.981405 28547 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:34:58.981518 28547 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:34:58.981774 28547 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.194
I20260811 06:34:58.991223 28547 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:40573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429755927508-29631-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429755927508-29631-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.239.194
--webserver_port=0
--tserver_master_addrs=127.28.239.254:37995
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.194
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429755927508-29631-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:34:58.992606 28547 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:34:58.994526 28547 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:34:59.008636 28557 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:34:59.008549 28555 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:34:59.009121 28559 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:34:59.009765 28547 server_base.cc:1034] running on GCE node
I20260811 06:34:59.010455 28547 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:34:59.011451 28547 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:34:59.012696 28547 hybrid_clock.cc:648] HybridClock initialized: now 1786430099012647 us; error 48 us; skew 500 ppm
I20260811 06:34:59.013015 28547 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:34:59.015662 28547 webserver.cc:460] Webserver started at http://127.28.239.194:36193/ using document root <none> and password file <none>
I20260811 06:34:59.016460 28547 fs_manager.cc:359] Metadata directory not provided
I20260811 06:34:59.016614 28547 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:34:59.017088 28547 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:34:59.020581 28547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429755927508-29631-0/minicluster-data/ts-1/data/instance:
uuid: "fb6aef4448cd4ef5b99c80c0a4f04067"
format_stamp: "Formatted at 2026-08-11 06:34:59 on dist-test-slave-gkw7"
server_key: "9a1031432377d28622dfac3c4d216045"
server_key_iv: "b092cbd5ca2244b709f51be33ef4fc43"
server_key_version: "encryptionkey@0"
I20260811 06:34:59.021413 28547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429755927508-29631-0/minicluster-data/ts-1/wal/instance:
uuid: "fb6aef4448cd4ef5b99c80c0a4f04067"
format_stamp: "Formatted at 2026-08-11 06:34:59 on dist-test-slave-gkw7"
server_key: "9a1031432377d28622dfac3c4d216045"
server_key_iv: "b092cbd5ca2244b709f51be33ef4fc43"
server_key_version: "encryptionkey@0"
I20260811 06:34:59.027297 28547 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 06:34:59.031255 28569 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:34:59.032667 28547 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 06:34:59.032826 28547 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429755927508-29631-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429755927508-29631-0/minicluster-data/ts-1/wal
uuid: "fb6aef4448cd4ef5b99c80c0a4f04067"
format_stamp: "Formatted at 2026-08-11 06:34:59 on dist-test-slave-gkw7"
server_key: "9a1031432377d28622dfac3c4d216045"
server_key_iv: "b092cbd5ca2244b709f51be33ef4fc43"
server_key_version: "encryptionkey@0"
I20260811 06:34:59.033023 28547 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429755927508-29631-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429755927508-29631-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429755927508-29631-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:34:59.066396 28547 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:34:59.067348 28547 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:34:59.067649 28547 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:34:59.068691 28547 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:34:59.070305 28547 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:34:59.070399 28547 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:34:59.070487 28547 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:34:59.070542 28547 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:34:59.106518 28547 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.194:44831
I20260811 06:34:59.106634 28726 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.194:44831 every 8 connection(s)
I20260811 06:34:59.107970 28547 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429755927508-29631-0/minicluster-data/ts-1/data/info.pb
I20260811 06:34:59.112658 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 28547
I20260811 06:34:59.112902 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429755927508-29631-0/minicluster-data/ts-1/wal/instance
I20260811 06:34:59.113400 29631 external_mini_cluster.cc:1428] Setting key b03a1b69095df8ac08f58616670b4a6f
I20260811 06:34:59.116705 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429755927508-29631-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429755927508-29631-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429755927508-29631-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.239.195:0
--local_ip_for_outbound_sockets=127.28.239.195
--webserver_interface=127.28.239.195
--webserver_port=0
--tserver_master_addrs=127.28.239.254:37995
--builtin_ntp_servers=127.28.239.212:40573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 06:34:59.122155 28544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:59.110929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.194:54591 (local address 127.28.239.254:37995)
0811 06:34:59.111135 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 06:34:59.111139 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:34:59.112690 (+ 1551us) server_negotiation.cc:407] Connection header received
0811 06:34:59.115116 (+ 2426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:34:59.115120 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:34:59.115187 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:34:59.115277 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:34:59.116586 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:59.117401 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:59.119194 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:59.119617 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:59.119765 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:59.121225 (+ 1460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:34:59.121230 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:34:59.121233 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:34:59.121253 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:34:59.121287 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:34:59.121299 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:34:59.121400 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 06:34:59.121684 (+ 284us) server_negotiation.cc:299] Negotiation successful
0811 06:34:59.121806 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 06:34:59.123318 28730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:59.111529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:37995 (local address 127.28.239.194:54591)
0811 06:34:59.112508 (+ 979us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:59.112554 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 06:34:59.114800 (+ 2246us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:34:59.115616 (+ 816us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:34:59.115628 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:34:59.115772 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:34:59.116406 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:59.116424 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:59.117600 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:59.117604 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:59.119036 (+ 1432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:59.119063 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:59.119863 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:59.119872 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:59.120113 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:59.120875 (+ 762us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 06:34:59.120878 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:34:59.120923 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 06:34:59.121008 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:34:59.121401 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:34:59.121415 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:34:59.121426 (+ 11us) client_negotiation.cc:769] Sending connection context
0811 06:34:59.121676 (+ 250us) client_negotiation.cc:240] Negotiation successful
0811 06:34:59.122225 (+ 549us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":286,"threads_started":1}
I20260811 06:34:59.124881 28727 heartbeater.cc:347] Connected to a master server at 127.28.239.254:37995
I20260811 06:34:59.125206 28727 heartbeater.cc:464] Registering TS with master...
I20260811 06:34:59.126197 28727 heartbeater.cc:511] Master 127.28.239.254:37995 requested a full tablet report, sending...
I20260811 06:34:59.127836 28261 ts_manager.cc:194] Registered new tserver with Master: fb6aef4448cd4ef5b99c80c0a4f04067 (127.28.239.194:44831)
I20260811 06:34:59.129108 28261 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.28.239.194:54591
W20260811 06:34:59.478374 28735 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:34:59.478678 28735 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:34:59.478765 28735 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:34:59.486050 28735 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:34:59.486159 28735 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:34:59.486274 28735 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.239.195
I20260811 06:34:59.495820 28735 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:40573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429755927508-29631-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429755927508-29631-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.239.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.239.195
--webserver_port=0
--tserver_master_addrs=127.28.239.254:37995
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.239.195
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429755927508-29631-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:34:59.497224 28735 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:34:59.499135 28735 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:34:59.513079 28740 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:34:59.513043 28744 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:34:59.513010 28741 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:34:59.514257 28735 server_base.cc:1034] running on GCE node
I20260811 06:34:59.515051 28735 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:34:59.516103 28735 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:34:59.517347 28735 hybrid_clock.cc:648] HybridClock initialized: now 1786430099517306 us; error 40 us; skew 500 ppm
I20260811 06:34:59.517765 28735 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:34:59.523972 28735 webserver.cc:460] Webserver started at http://127.28.239.195:35579/ using document root <none> and password file <none>
I20260811 06:34:59.524792 28735 fs_manager.cc:359] Metadata directory not provided
I20260811 06:34:59.524941 28735 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:34:59.525342 28735 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:34:59.528592 28735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429755927508-29631-0/minicluster-data/ts-2/data/instance:
uuid: "9084db42d92e4d99a500de07ecdb741c"
format_stamp: "Formatted at 2026-08-11 06:34:59 on dist-test-slave-gkw7"
server_key: "61a2c28a97385fe9a9d092db68b8701f"
server_key_iv: "740b6b3152af4bacdb2024f04961bf6c"
server_key_version: "encryptionkey@0"
I20260811 06:34:59.529524 28735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429755927508-29631-0/minicluster-data/ts-2/wal/instance:
uuid: "9084db42d92e4d99a500de07ecdb741c"
format_stamp: "Formatted at 2026-08-11 06:34:59 on dist-test-slave-gkw7"
server_key: "61a2c28a97385fe9a9d092db68b8701f"
server_key_iv: "740b6b3152af4bacdb2024f04961bf6c"
server_key_version: "encryptionkey@0"
I20260811 06:34:59.535066 28735 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 06:34:59.538893 28753 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:34:59.540354 28735 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260811 06:34:59.540549 28735 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429755927508-29631-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429755927508-29631-0/minicluster-data/ts-2/wal
uuid: "9084db42d92e4d99a500de07ecdb741c"
format_stamp: "Formatted at 2026-08-11 06:34:59 on dist-test-slave-gkw7"
server_key: "61a2c28a97385fe9a9d092db68b8701f"
server_key_iv: "740b6b3152af4bacdb2024f04961bf6c"
server_key_version: "encryptionkey@0"
I20260811 06:34:59.540789 28735 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429755927508-29631-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429755927508-29631-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429755927508-29631-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:34:59.572501 28735 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:34:59.573485 28735 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:34:59.573880 28735 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:34:59.574999 28735 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:34:59.576660 28735 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:34:59.576799 28735 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:34:59.576946 28735 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:34:59.577042 28735 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:34:59.613747 28735 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.195:42081
I20260811 06:34:59.613809 28932 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.195:42081 every 8 connection(s)
I20260811 06:34:59.615231 28735 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429755927508-29631-0/minicluster-data/ts-2/data/info.pb
I20260811 06:34:59.623245 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 28735
I20260811 06:34:59.623471 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429755927508-29631-0/minicluster-data/ts-2/wal/instance
I20260811 06:34:59.623929 29631 external_mini_cluster.cc:1428] Setting key 4b88e8a0bd1275c383fab8f142925a35
I20260811 06:34:59.629369 28544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:59.618163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.195:47129 (local address 127.28.239.254:37995)
0811 06:34:59.618381 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0811 06:34:59.618386 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:34:59.619983 (+ 1597us) server_negotiation.cc:407] Connection header received
0811 06:34:59.621974 (+ 1991us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:34:59.621980 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:34:59.622091 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:34:59.622220 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:34:59.623416 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:59.624445 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:59.626391 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:59.626833 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:59.626981 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:59.628405 (+ 1424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:34:59.628411 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:34:59.628437 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:34:59.628464 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:34:59.628506 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:34:59.628522 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:34:59.628653 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 06:34:59.628909 (+ 256us) server_negotiation.cc:299] Negotiation successful
0811 06:34:59.629014 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260811 06:34:59.630529 28938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:59.618786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:37995 (local address 127.28.239.195:47129)
0811 06:34:59.619804 (+ 1018us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:59.619848 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 06:34:59.621670 (+ 1822us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:34:59.622479 (+ 809us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:34:59.622488 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:34:59.622585 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:34:59.623229 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:59.623247 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:59.624651 (+ 1404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:59.624657 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:59.626228 (+ 1571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:59.626249 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:59.627072 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:59.627079 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:59.627340 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:59.628046 (+ 706us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 06:34:59.628048 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:34:59.628091 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 06:34:59.628187 (+ 96us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:34:59.628651 (+ 464us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:34:59.628665 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:34:59.628675 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 06:34:59.628877 (+ 202us) client_negotiation.cc:240] Negotiation successful
0811 06:34:59.629307 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":290,"threads_started":1}
I20260811 06:34:59.631768 28542 heartbeater.cc:503] Master 127.28.239.254:37995 was elected leader, sending a full tablet report...
I20260811 06:34:59.632100 28935 heartbeater.cc:347] Connected to a master server at 127.28.239.254:37995
I20260811 06:34:59.632463 28935 heartbeater.cc:464] Registering TS with master...
I20260811 06:34:59.633412 28935 heartbeater.cc:511] Master 127.28.239.254:37995 requested a full tablet report, sending...
I20260811 06:34:59.634960 28261 ts_manager.cc:194] Registered new tserver with Master: 9084db42d92e4d99a500de07ecdb741c (127.28.239.195:42081)
I20260811 06:34:59.635718 28261 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.28.239.195:47129
I20260811 06:34:59.639449 29631 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 06:34:59.650205 28544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:59.644071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56730 (local address 127.28.239.254:37995)
0811 06:34:59.644416 (+ 345us) server_negotiation.cc:206] Beginning negotiation
0811 06:34:59.644422 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:34:59.644765 (+ 343us) server_negotiation.cc:407] Connection header received
0811 06:34:59.644901 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:34:59.644906 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:34:59.644998 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:34:59.645196 (+ 198us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:34:59.645692 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:59.646425 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:59.647766 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:59.648169 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:59.648265 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:59.649415 (+ 1150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:34:59.649420 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:34:59.649423 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:34:59.649443 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:34:59.649480 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:34:59.649493 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:34:59.649716 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0811 06:34:59.649854 (+ 138us) server_negotiation.cc:299] Negotiation successful
0811 06:34:59.649938 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124}
I20260811 06:34:59.654433 28261 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56730:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:34:59.661350 28261 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:34:59.698310 28954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:59.674183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.194:44831 (local address 127.0.0.1:53686)
0811 06:34:59.686219 (+ 12036us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:59.686297 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0811 06:34:59.686554 (+ 257us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:34:59.689925 (+ 3371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:34:59.689938 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:34:59.689988 (+ 50us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:34:59.690866 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:59.690881 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:59.692882 (+ 2001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:59.692888 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:59.695479 (+ 2591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:59.695490 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:59.697234 (+ 1744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:59.697243 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:59.697462 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:59.697523 (+ 61us) client_negotiation.cc:769] Sending connection context
0811 06:34:59.697697 (+ 174us) client_negotiation.cc:240] Negotiation successful
0811 06:34:59.697920 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11663,"mutex_wait_us":79,"thread_start_us":11475,"threads_started":1}
I20260811 06:34:59.699232 28960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:59.674961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53686 (local address 127.28.239.194:44831)
0811 06:34:59.688878 (+ 13917us) server_negotiation.cc:206] Beginning negotiation
0811 06:34:59.688891 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 06:34:59.688934 (+ 43us) server_negotiation.cc:407] Connection header received
0811 06:34:59.689082 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:34:59.689095 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:34:59.689472 (+ 377us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:34:59.690139 (+ 667us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:34:59.691077 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:59.692731 (+ 1654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:59.695693 (+ 2962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:59.697012 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:59.697205 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:59.697473 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0811 06:34:59.697766 (+ 293us) server_negotiation.cc:299] Negotiation successful
0811 06:34:59.697974 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13698,"thread_start_us":13544,"threads_started":1}
I20260811 06:34:59.699492 28957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:59.674278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.193:43215 (local address 127.0.0.1:42186)
0811 06:34:59.687275 (+ 12997us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:59.687304 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0811 06:34:59.687428 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:34:59.691231 (+ 3803us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:34:59.691236 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:34:59.691263 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:34:59.691679 (+ 416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:59.691692 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:59.693553 (+ 1861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:59.693560 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:59.696053 (+ 2493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:59.696069 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:59.698475 (+ 2406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:59.698481 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:59.698694 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:59.698740 (+ 46us) client_negotiation.cc:769] Sending connection context
0811 06:34:59.698829 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 06:34:59.698966 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12804,"mutex_wait_us":104,"thread_start_us":13233,"threads_started":1}
I20260811 06:34:59.700132 28958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:59.674415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42186 (local address 127.28.239.193:43215)
0811 06:34:59.690547 (+ 16132us) server_negotiation.cc:206] Beginning negotiation
0811 06:34:59.690555 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:34:59.690585 (+ 30us) server_negotiation.cc:407] Connection header received
0811 06:34:59.690688 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:34:59.690699 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:34:59.690950 (+ 251us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:34:59.691917 (+ 967us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:34:59.692155 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:59.693376 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:59.696295 (+ 2919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:59.697860 (+ 1565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:59.698061 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:59.698342 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0811 06:34:59.699379 (+ 1037us) server_negotiation.cc:299] Negotiation successful
0811 06:34:59.699775 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15937,"thread_start_us":15771,"threads_started":1}
I20260811 06:34:59.700356 28959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:59.674650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.195:42081 (local address 127.0.0.1:59462)
0811 06:34:59.691178 (+ 16528us) negotiation.cc:107] Waiting for socket to connect
0811 06:34:59.691204 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 06:34:59.691432 (+ 228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:34:59.692166 (+ 734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:34:59.692170 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:34:59.692195 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:34:59.692550 (+ 355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:59.692560 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:59.694580 (+ 2020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:59.694589 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:59.696866 (+ 2277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:34:59.696879 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:34:59.699194 (+ 2315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:34:59.699200 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:34:59.699733 (+ 533us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:59.699765 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 06:34:59.699856 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 06:34:59.699989 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16390,"mutex_wait_us":92,"thread_start_us":15062,"threads_started":1}
I20260811 06:34:59.701320 28961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:34:59.675306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59462 (local address 127.28.239.195:42081)
0811 06:34:59.688311 (+ 13005us) server_negotiation.cc:206] Beginning negotiation
0811 06:34:59.688325 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 06:34:59.691308 (+ 2983us) server_negotiation.cc:407] Connection header received
0811 06:34:59.691682 (+ 374us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:34:59.691689 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:34:59.691967 (+ 278us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:34:59.692179 (+ 212us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:34:59.692724 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:59.694393 (+ 1669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:59.697052 (+ 2659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:34:59.698592 (+ 1540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:34:59.698788 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:34:59.699052 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0811 06:34:59.700736 (+ 1684us) server_negotiation.cc:299] Negotiation successful
0811 06:34:59.700922 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12769,"thread_start_us":12592,"threads_started":1}
I20260811 06:34:59.703635 28453 tablet_service.cc:1467] Processing CreateTablet for tablet ef9036ca44364222b5911d9474f24755 (DEFAULT_TABLE table=test-table [id=44220b37addd4c089d55a925ed7d958a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:34:59.704933 28453 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef9036ca44364222b5911d9474f24755. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:34:59.703675 28638 tablet_service.cc:1467] Processing CreateTablet for tablet ef9036ca44364222b5911d9474f24755 (DEFAULT_TABLE table=test-table [id=44220b37addd4c089d55a925ed7d958a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:34:59.705156 28835 tablet_service.cc:1467] Processing CreateTablet for tablet ef9036ca44364222b5911d9474f24755 (DEFAULT_TABLE table=test-table [id=44220b37addd4c089d55a925ed7d958a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:34:59.706643 28638 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef9036ca44364222b5911d9474f24755. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:34:59.707413 28835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef9036ca44364222b5911d9474f24755. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:34:59.717065 28962 tablet_bootstrap.cc:492] T ef9036ca44364222b5911d9474f24755 P 29e6eba07d584b298acefddfc15f40ba: Bootstrap starting.
I20260811 06:34:59.720872 28962 tablet_bootstrap.cc:654] T ef9036ca44364222b5911d9474f24755 P 29e6eba07d584b298acefddfc15f40ba: Neither blocks nor log segments found. Creating new log.
I20260811 06:34:59.722154 28962 log.cc:824] T ef9036ca44364222b5911d9474f24755 P 29e6eba07d584b298acefddfc15f40ba: Log is configured to *not* fsync() on all Append() calls
I20260811 06:34:59.723135 28967 tablet_bootstrap.cc:492] T ef9036ca44364222b5911d9474f24755 P fb6aef4448cd4ef5b99c80c0a4f04067: Bootstrap starting.
I20260811 06:34:59.723721 28966 tablet_bootstrap.cc:492] T ef9036ca44364222b5911d9474f24755 P 9084db42d92e4d99a500de07ecdb741c: Bootstrap starting.
I20260811 06:34:59.726311 28962 tablet_bootstrap.cc:492] T ef9036ca44364222b5911d9474f24755 P 29e6eba07d584b298acefddfc15f40ba: No bootstrap required, opened a new log
I20260811 06:34:59.726600 28962 ts_tablet_manager.cc:1397] T ef9036ca44364222b5911d9474f24755 P 29e6eba07d584b298acefddfc15f40ba: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.005s
I20260811 06:34:59.727594 28967 tablet_bootstrap.cc:654] T ef9036ca44364222b5911d9474f24755 P fb6aef4448cd4ef5b99c80c0a4f04067: Neither blocks nor log segments found. Creating new log.
I20260811 06:34:59.727643 28966 tablet_bootstrap.cc:654] T ef9036ca44364222b5911d9474f24755 P 9084db42d92e4d99a500de07ecdb741c: Neither blocks nor log segments found. Creating new log.
I20260811 06:34:59.728995 28967 log.cc:824] T ef9036ca44364222b5911d9474f24755 P fb6aef4448cd4ef5b99c80c0a4f04067: Log is configured to *not* fsync() on all Append() calls
I20260811 06:34:59.729092 28966 log.cc:824] T ef9036ca44364222b5911d9474f24755 P 9084db42d92e4d99a500de07ecdb741c: Log is configured to *not* fsync() on all Append() calls
I20260811 06:34:59.732697 28966 tablet_bootstrap.cc:492] T ef9036ca44364222b5911d9474f24755 P 9084db42d92e4d99a500de07ecdb741c: No bootstrap required, opened a new log
I20260811 06:34:59.733017 28966 ts_tablet_manager.cc:1397] T ef9036ca44364222b5911d9474f24755 P 9084db42d92e4d99a500de07ecdb741c: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.006s
I20260811 06:34:59.733477 28967 tablet_bootstrap.cc:492] T ef9036ca44364222b5911d9474f24755 P fb6aef4448cd4ef5b99c80c0a4f04067: No bootstrap required, opened a new log
I20260811 06:34:59.733803 28967 ts_tablet_manager.cc:1397] T ef9036ca44364222b5911d9474f24755 P fb6aef4448cd4ef5b99c80c0a4f04067: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.006s
I20260811 06:34:59.736959 28962 raft_consensus.cc:348] T ef9036ca44364222b5911d9474f24755 P 29e6eba07d584b298acefddfc15f40ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29e6eba07d584b298acefddfc15f40ba" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 43215 } } peers { permanent_uuid: "fb6aef4448cd4ef5b99c80c0a4f04067" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 44831 } } peers { permanent_uuid: "9084db42d92e4d99a500de07ecdb741c" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 42081 } }
I20260811 06:34:59.737430 28962 raft_consensus.cc:374] T ef9036ca44364222b5911d9474f24755 P 29e6eba07d584b298acefddfc15f40ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:34:59.737577 28962 raft_consensus.cc:729] T ef9036ca44364222b5911d9474f24755 P 29e6eba07d584b298acefddfc15f40ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29e6eba07d584b298acefddfc15f40ba, State: Initialized, Role: FOLLOWER
I20260811 06:34:59.738334 28962 consensus_queue.cc:260] T ef9036ca44364222b5911d9474f24755 P 29e6eba07d584b298acefddfc15f40ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29e6eba07d584b298acefddfc15f40ba" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 43215 } } peers { permanent_uuid: "fb6aef4448cd4ef5b99c80c0a4f04067" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 44831 } } peers { permanent_uuid: "9084db42d92e4d99a500de07ecdb741c" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 42081 } }
I20260811 06:34:59.744383 28966 raft_consensus.cc:348] T ef9036ca44364222b5911d9474f24755 P 9084db42d92e4d99a500de07ecdb741c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29e6eba07d584b298acefddfc15f40ba" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 43215 } } peers { permanent_uuid: "fb6aef4448cd4ef5b99c80c0a4f04067" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 44831 } } peers { permanent_uuid: "9084db42d92e4d99a500de07ecdb741c" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 42081 } }
I20260811 06:34:59.744947 28966 raft_consensus.cc:374] T ef9036ca44364222b5911d9474f24755 P 9084db42d92e4d99a500de07ecdb741c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:34:59.745102 28966 raft_consensus.cc:729] T ef9036ca44364222b5911d9474f24755 P 9084db42d92e4d99a500de07ecdb741c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9084db42d92e4d99a500de07ecdb741c, State: Initialized, Role: FOLLOWER
I20260811 06:34:59.745060 28967 raft_consensus.cc:348] T ef9036ca44364222b5911d9474f24755 P fb6aef4448cd4ef5b99c80c0a4f04067 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29e6eba07d584b298acefddfc15f40ba" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 43215 } } peers { permanent_uuid: "fb6aef4448cd4ef5b99c80c0a4f04067" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 44831 } } peers { permanent_uuid: "9084db42d92e4d99a500de07ecdb741c" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 42081 } }
I20260811 06:34:59.745492 28967 raft_consensus.cc:374] T ef9036ca44364222b5911d9474f24755 P fb6aef4448cd4ef5b99c80c0a4f04067 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:34:59.745621 28967 raft_consensus.cc:729] T ef9036ca44364222b5911d9474f24755 P fb6aef4448cd4ef5b99c80c0a4f04067 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb6aef4448cd4ef5b99c80c0a4f04067, State: Initialized, Role: FOLLOWER
I20260811 06:34:59.745860 28966 consensus_queue.cc:260] T ef9036ca44364222b5911d9474f24755 P 9084db42d92e4d99a500de07ecdb741c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29e6eba07d584b298acefddfc15f40ba" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 43215 } } peers { permanent_uuid: "fb6aef4448cd4ef5b99c80c0a4f04067" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 44831 } } peers { permanent_uuid: "9084db42d92e4d99a500de07ecdb741c" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 42081 } }
I20260811 06:34:59.746342 28967 consensus_queue.cc:260] T ef9036ca44364222b5911d9474f24755 P fb6aef4448cd4ef5b99c80c0a4f04067 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29e6eba07d584b298acefddfc15f40ba" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 43215 } } peers { permanent_uuid: "fb6aef4448cd4ef5b99c80c0a4f04067" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 44831 } } peers { permanent_uuid: "9084db42d92e4d99a500de07ecdb741c" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 42081 } }
I20260811 06:34:59.747255 28935 heartbeater.cc:503] Master 127.28.239.254:37995 was elected leader, sending a full tablet report...
I20260811 06:34:59.747956 28966 ts_tablet_manager.cc:1428] T ef9036ca44364222b5911d9474f24755 P 9084db42d92e4d99a500de07ecdb741c: Time spent starting tablet: real 0.015s user 0.007s sys 0.009s
I20260811 06:34:59.749898 28727 heartbeater.cc:503] Master 127.28.239.254:37995 was elected leader, sending a full tablet report...
I20260811 06:34:59.750597 28967 ts_tablet_manager.cc:1428] T ef9036ca44364222b5911d9474f24755 P fb6aef4448cd4ef5b99c80c0a4f04067: Time spent starting tablet: real 0.017s user 0.008s sys 0.006s
I20260811 06:34:59.751645 28962 ts_tablet_manager.cc:1428] T ef9036ca44364222b5911d9474f24755 P 29e6eba07d584b298acefddfc15f40ba: Time spent starting tablet: real 0.025s user 0.015s sys 0.006s
W20260811 06:34:59.856448 28543 tablet.cc:2369] T ef9036ca44364222b5911d9474f24755 P 29e6eba07d584b298acefddfc15f40ba: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 06:34:59.860297 28728 tablet.cc:2369] T ef9036ca44364222b5911d9474f24755 P fb6aef4448cd4ef5b99c80c0a4f04067: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 06:34:59.867129 28937 tablet.cc:2369] T ef9036ca44364222b5911d9474f24755 P 9084db42d92e4d99a500de07ecdb741c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:35:00.031330 28975 raft_consensus.cc:482] T ef9036ca44364222b5911d9474f24755 P fb6aef4448cd4ef5b99c80c0a4f04067 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:35:00.031723 28975 raft_consensus.cc:504] T ef9036ca44364222b5911d9474f24755 P fb6aef4448cd4ef5b99c80c0a4f04067 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29e6eba07d584b298acefddfc15f40ba" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 43215 } } peers { permanent_uuid: "fb6aef4448cd4ef5b99c80c0a4f04067" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 44831 } } peers { permanent_uuid: "9084db42d92e4d99a500de07ecdb741c" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 42081 } }
I20260811 06:35:00.033903 28975 leader_election.cc:288] T ef9036ca44364222b5911d9474f24755 P fb6aef4448cd4ef5b99c80c0a4f04067 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 29e6eba07d584b298acefddfc15f40ba (127.28.239.193:43215), 9084db42d92e4d99a500de07ecdb741c (127.28.239.195:42081)
I20260811 06:35:00.042053 28958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:00.034489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.194:56221 (local address 127.28.239.193:43215)
0811 06:35:00.034730 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:00.034739 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:00.035475 (+ 736us) server_negotiation.cc:407] Connection header received
0811 06:35:00.035801 (+ 326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:00.035806 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:00.035904 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:00.036006 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:35:00.036663 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:00.037658 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:00.040028 (+ 2370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:00.041053 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:00.041263 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:00.041344 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 06:35:00.041654 (+ 310us) server_negotiation.cc:299] Negotiation successful
0811 06:35:00.041791 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260811 06:35:00.042160 28977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:00.034472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.193:43215 (local address 127.28.239.194:56221)
0811 06:35:00.035344 (+ 872us) negotiation.cc:107] Waiting for socket to connect
0811 06:35:00.035361 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 06:35:00.035555 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:35:00.036036 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:35:00.036042 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:35:00.036063 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:35:00.036436 (+ 373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:35:00.036446 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:35:00.037795 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:35:00.037802 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:35:00.039815 (+ 2013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:35:00.039829 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:35:00.041282 (+ 1453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:35:00.041288 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:35:00.041463 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:00.041488 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 06:35:00.041625 (+ 137us) client_negotiation.cc:240] Negotiation successful
0811 06:35:00.041762 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":727,"mutex_wait_us":3,"thread_start_us":496,"threads_started":1}
I20260811 06:35:00.042800 28979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:00.034533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.195:42081 (local address 127.28.239.194:45925)
0811 06:35:00.035519 (+ 986us) negotiation.cc:107] Waiting for socket to connect
0811 06:35:00.035543 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 06:35:00.035691 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:35:00.036428 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:35:00.036431 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:35:00.036453 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:35:00.036901 (+ 448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:35:00.036914 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:35:00.038741 (+ 1827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:35:00.038745 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:35:00.040330 (+ 1585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:35:00.040346 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:35:00.041901 (+ 1555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:35:00.041914 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:35:00.042194 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:00.042307 (+ 113us) client_negotiation.cc:769] Sending connection context
0811 06:35:00.042409 (+ 102us) client_negotiation.cc:240] Negotiation successful
0811 06:35:00.042551 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":850,"thread_start_us":594,"threads_started":1}
I20260811 06:35:00.042869 28961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:00.034423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.239.194:45925 (local address 127.28.239.195:42081)
0811 06:35:00.034658 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:00.034666 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:00.035657 (+ 991us) server_negotiation.cc:407] Connection header received
0811 06:35:00.035842 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:00.035847 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:00.035917 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:00.036004 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:35:00.037163 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:00.038582 (+ 1419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:00.040559 (+ 1977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:00.041594 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:00.041811 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:00.041892 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 06:35:00.042465 (+ 573us) server_negotiation.cc:299] Negotiation successful
0811 06:35:00.042619 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 06:35:00.043483 28485 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef9036ca44364222b5911d9474f24755" candidate_uuid: "fb6aef4448cd4ef5b99c80c0a4f04067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29e6eba07d584b298acefddfc15f40ba" is_pre_election: true
I20260811 06:35:00.043834 28869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef9036ca44364222b5911d9474f24755" candidate_uuid: "fb6aef4448cd4ef5b99c80c0a4f04067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9084db42d92e4d99a500de07ecdb741c" is_pre_election: true
I20260811 06:35:00.044148 28485 raft_consensus.cc:2454] T ef9036ca44364222b5911d9474f24755 P 29e6eba07d584b298acefddfc15f40ba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb6aef4448cd4ef5b99c80c0a4f04067 in term 0.
I20260811 06:35:00.044457 28869 raft_consensus.cc:2454] T ef9036ca44364222b5911d9474f24755 P 9084db42d92e4d99a500de07ecdb741c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb6aef4448cd4ef5b99c80c0a4f04067 in term 0.
I20260811 06:35:00.045007 28570 leader_election.cc:302] T ef9036ca44364222b5911d9474f24755 P fb6aef4448cd4ef5b99c80c0a4f04067 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29e6eba07d584b298acefddfc15f40ba, fb6aef4448cd4ef5b99c80c0a4f04067; no voters:
I20260811 06:35:00.045454 28975 raft_consensus.cc:2781] T ef9036ca44364222b5911d9474f24755 P fb6aef4448cd4ef5b99c80c0a4f04067 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:35:00.045683 28975 raft_consensus.cc:482] T ef9036ca44364222b5911d9474f24755 P fb6aef4448cd4ef5b99c80c0a4f04067 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:35:00.045812 28975 raft_consensus.cc:3037] T ef9036ca44364222b5911d9474f24755 P fb6aef4448cd4ef5b99c80c0a4f04067 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:35:00.048313 28975 raft_consensus.cc:504] T ef9036ca44364222b5911d9474f24755 P fb6aef4448cd4ef5b99c80c0a4f04067 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29e6eba07d584b298acefddfc15f40ba" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 43215 } } peers { permanent_uuid: "fb6aef4448cd4ef5b99c80c0a4f04067" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 44831 } } peers { permanent_uuid: "9084db42d92e4d99a500de07ecdb741c" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 42081 } }
I20260811 06:35:00.049045 28975 leader_election.cc:288] T ef9036ca44364222b5911d9474f24755 P fb6aef4448cd4ef5b99c80c0a4f04067 [CANDIDATE]: Term 1 election: Requested vote from peers 29e6eba07d584b298acefddfc15f40ba (127.28.239.193:43215), 9084db42d92e4d99a500de07ecdb741c (127.28.239.195:42081)
I20260811 06:35:00.049676 28869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef9036ca44364222b5911d9474f24755" candidate_uuid: "fb6aef4448cd4ef5b99c80c0a4f04067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9084db42d92e4d99a500de07ecdb741c"
I20260811 06:35:00.049711 28485 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef9036ca44364222b5911d9474f24755" candidate_uuid: "fb6aef4448cd4ef5b99c80c0a4f04067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29e6eba07d584b298acefddfc15f40ba"
I20260811 06:35:00.049911 28869 raft_consensus.cc:3037] T ef9036ca44364222b5911d9474f24755 P 9084db42d92e4d99a500de07ecdb741c [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:35:00.049939 28485 raft_consensus.cc:3037] T ef9036ca44364222b5911d9474f24755 P 29e6eba07d584b298acefddfc15f40ba [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:35:00.052270 28869 raft_consensus.cc:2454] T ef9036ca44364222b5911d9474f24755 P 9084db42d92e4d99a500de07ecdb741c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb6aef4448cd4ef5b99c80c0a4f04067 in term 1.
I20260811 06:35:00.052459 28485 raft_consensus.cc:2454] T ef9036ca44364222b5911d9474f24755 P 29e6eba07d584b298acefddfc15f40ba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb6aef4448cd4ef5b99c80c0a4f04067 in term 1.
I20260811 06:35:00.052783 28572 leader_election.cc:302] T ef9036ca44364222b5911d9474f24755 P fb6aef4448cd4ef5b99c80c0a4f04067 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9084db42d92e4d99a500de07ecdb741c, fb6aef4448cd4ef5b99c80c0a4f04067; no voters:
I20260811 06:35:00.053181 28975 raft_consensus.cc:2781] T ef9036ca44364222b5911d9474f24755 P fb6aef4448cd4ef5b99c80c0a4f04067 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:35:00.053766 28975 raft_consensus.cc:686] T ef9036ca44364222b5911d9474f24755 P fb6aef4448cd4ef5b99c80c0a4f04067 [term 1 LEADER]: Becoming Leader. State: Replica: fb6aef4448cd4ef5b99c80c0a4f04067, State: Running, Role: LEADER
I20260811 06:35:00.054354 28975 consensus_queue.cc:237] T ef9036ca44364222b5911d9474f24755 P fb6aef4448cd4ef5b99c80c0a4f04067 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29e6eba07d584b298acefddfc15f40ba" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 43215 } } peers { permanent_uuid: "fb6aef4448cd4ef5b99c80c0a4f04067" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 44831 } } peers { permanent_uuid: "9084db42d92e4d99a500de07ecdb741c" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 42081 } }
I20260811 06:35:00.060106 28259 catalog_manager.cc:5495] T ef9036ca44364222b5911d9474f24755 P fb6aef4448cd4ef5b99c80c0a4f04067 reported cstate change: term changed from 0 to 1, leader changed from <none> to fb6aef4448cd4ef5b99c80c0a4f04067 (127.28.239.194). New cstate: current_term: 1 leader_uuid: "fb6aef4448cd4ef5b99c80c0a4f04067" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29e6eba07d584b298acefddfc15f40ba" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 43215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fb6aef4448cd4ef5b99c80c0a4f04067" member_type: VOTER last_known_addr { host: "127.28.239.194" port: 44831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9084db42d92e4d99a500de07ecdb741c" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 42081 } health_report { overall_health: UNKNOWN } } }
I20260811 06:35:00.158934 28960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:00.152470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53692 (local address 127.28.239.194:44831)
0811 06:35:00.152682 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:00.152687 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:00.153133 (+ 446us) server_negotiation.cc:407] Connection header received
0811 06:35:00.153323 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:00.153327 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:00.153428 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:00.153649 (+ 221us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:35:00.154080 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:00.154806 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:00.156458 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:00.156893 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:00.156988 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:00.157700 (+ 712us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:35:00.158288 (+ 588us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:35:00.158420 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 06:35:00.158581 (+ 161us) server_negotiation.cc:299] Negotiation successful
0811 06:35:00.158654 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 06:35:00.167620 28869 raft_consensus.cc:1261] T ef9036ca44364222b5911d9474f24755 P 9084db42d92e4d99a500de07ecdb741c [term 1 FOLLOWER]: Refusing update from remote peer fb6aef4448cd4ef5b99c80c0a4f04067: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:35:00.167755 28485 raft_consensus.cc:1261] T ef9036ca44364222b5911d9474f24755 P 29e6eba07d584b298acefddfc15f40ba [term 1 FOLLOWER]: Refusing update from remote peer fb6aef4448cd4ef5b99c80c0a4f04067: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:35:00.168751 28981 consensus_queue.cc:1035] T ef9036ca44364222b5911d9474f24755 P fb6aef4448cd4ef5b99c80c0a4f04067 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9084db42d92e4d99a500de07ecdb741c" member_type: VOTER last_known_addr { host: "127.28.239.195" port: 42081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:35:00.169164 28975 consensus_queue.cc:1035] T ef9036ca44364222b5911d9474f24755 P fb6aef4448cd4ef5b99c80c0a4f04067 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29e6eba07d584b298acefddfc15f40ba" member_type: VOTER last_known_addr { host: "127.28.239.193" port: 43215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:35:00.181218 28986 mvcc.cc:205] Tried to move back new op lower bound from 7317217690275848192 to 7317217689827442688. Current Snapshot: MvccSnapshot[applied={T|T < 7317217690275848192}]
I20260811 06:35:00.194291 28261 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:56730:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:35:00.194643 28261 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:56730:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:35:00.198734 28261 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fa9d87218ab8425aa52f1b08aea67667: Sending DeleteTablet for 3 replicas of tablet ef9036ca44364222b5911d9474f24755
I20260811 06:35:00.200095 28638 tablet_service.cc:1514] Processing DeleteTablet for tablet ef9036ca44364222b5911d9474f24755 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:35:00 UTC) from {username='slave'} at 127.0.0.1:53686
I20260811 06:35:00.200088 28453 tablet_service.cc:1514] Processing DeleteTablet for tablet ef9036ca44364222b5911d9474f24755 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:35:00 UTC) from {username='slave'} at 127.0.0.1:42186
I20260811 06:35:00.200150 28835 tablet_service.cc:1514] Processing DeleteTablet for tablet ef9036ca44364222b5911d9474f24755 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:35:00 UTC) from {username='slave'} at 127.0.0.1:59462
I20260811 06:35:00.201356 29001 tablet_replica.cc:330] T ef9036ca44364222b5911d9474f24755 P fb6aef4448cd4ef5b99c80c0a4f04067: stopping tablet replica
I20260811 06:35:00.201960 29001 raft_consensus.cc:2229] T ef9036ca44364222b5911d9474f24755 P fb6aef4448cd4ef5b99c80c0a4f04067 [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:35:00.202173 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 28358
I20260811 06:35:00.202821 29001 raft_consensus.cc:2258] T ef9036ca44364222b5911d9474f24755 P fb6aef4448cd4ef5b99c80c0a4f04067 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:35:00.209012 29001 ts_tablet_manager.cc:1905] T ef9036ca44364222b5911d9474f24755 P fb6aef4448cd4ef5b99c80c0a4f04067: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:35:00.210075 29003 tablet_replica.cc:330] T ef9036ca44364222b5911d9474f24755 P 9084db42d92e4d99a500de07ecdb741c: stopping tablet replica
I20260811 06:35:00.210552 29003 raft_consensus.cc:2229] T ef9036ca44364222b5911d9474f24755 P 9084db42d92e4d99a500de07ecdb741c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:35:00.211097 29003 raft_consensus.cc:2258] T ef9036ca44364222b5911d9474f24755 P 9084db42d92e4d99a500de07ecdb741c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:35:00.211982 29002 tablet_replica.cc:330] T ef9036ca44364222b5911d9474f24755 P 29e6eba07d584b298acefddfc15f40ba: stopping tablet replica
I20260811 06:35:00.212433 29002 raft_consensus.cc:2229] T ef9036ca44364222b5911d9474f24755 P 29e6eba07d584b298acefddfc15f40ba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:35:00.212905 29002 raft_consensus.cc:2258] T ef9036ca44364222b5911d9474f24755 P 29e6eba07d584b298acefddfc15f40ba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:35:00.216084 29002 ts_tablet_manager.cc:1905] T ef9036ca44364222b5911d9474f24755 P 29e6eba07d584b298acefddfc15f40ba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:35:00.216148 29003 ts_tablet_manager.cc:1905] T ef9036ca44364222b5911d9474f24755 P 9084db42d92e4d99a500de07ecdb741c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:35:00.224299 29003 ts_tablet_manager.cc:1918] T ef9036ca44364222b5911d9474f24755 P 9084db42d92e4d99a500de07ecdb741c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:35:00.224318 29002 ts_tablet_manager.cc:1918] T ef9036ca44364222b5911d9474f24755 P 29e6eba07d584b298acefddfc15f40ba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:35:00.224764 29002 log.cc:1196] T ef9036ca44364222b5911d9474f24755 P 29e6eba07d584b298acefddfc15f40ba: Deleting WAL directory at /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429755927508-29631-0/minicluster-data/ts-0/wal/wals/ef9036ca44364222b5911d9474f24755
I20260811 06:35:00.225512 29002 ts_tablet_manager.cc:1939] T ef9036ca44364222b5911d9474f24755 P 29e6eba07d584b298acefddfc15f40ba: Deleting consensus metadata
I20260811 06:35:00.225600 28958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:00.205023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42188 (local address 127.28.239.193:43215)
0811 06:35:00.205229 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:00.205236 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:00.215975 (+ 10739us) server_negotiation.cc:407] Connection header received
0811 06:35:00.216249 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:00.216256 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:00.216388 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:00.216518 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:00.217108 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:00.218354 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:00.220651 (+ 2297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:00.221383 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:00.221502 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:00.223389 (+ 1887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:35:00.223402 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:35:00.223423 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:35:00.224379 (+ 956us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:35:00.224448 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:35:00.224478 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:35:00.224574 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 06:35:00.224876 (+ 302us) server_negotiation.cc:299] Negotiation successful
0811 06:35:00.225021 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 06:35:00.226966 28535 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:35:00.228318 29001 ts_tablet_manager.cc:1918] T ef9036ca44364222b5911d9474f24755 P fb6aef4448cd4ef5b99c80c0a4f04067: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:35:00.228515 29001 log.cc:1196] T ef9036ca44364222b5911d9474f24755 P fb6aef4448cd4ef5b99c80c0a4f04067: Deleting WAL directory at /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429755927508-29631-0/minicluster-data/ts-1/wal/wals/ef9036ca44364222b5911d9474f24755
I20260811 06:35:00.229135 29001 ts_tablet_manager.cc:1939] T ef9036ca44364222b5911d9474f24755 P fb6aef4448cd4ef5b99c80c0a4f04067: Deleting consensus metadata
I20260811 06:35:00.224506 29003 log.cc:1196] T ef9036ca44364222b5911d9474f24755 P 9084db42d92e4d99a500de07ecdb741c: Deleting WAL directory at /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786429755927508-29631-0/minicluster-data/ts-2/wal/wals/ef9036ca44364222b5911d9474f24755
I20260811 06:35:00.231027 29003 ts_tablet_manager.cc:1939] T ef9036ca44364222b5911d9474f24755 P 9084db42d92e4d99a500de07ecdb741c: Deleting consensus metadata
I20260811 06:35:00.233310 28246 catalog_manager.cc:4841] TS fb6aef4448cd4ef5b99c80c0a4f04067 (127.28.239.194:44831): tablet ef9036ca44364222b5911d9474f24755 (table test-table [id=44220b37addd4c089d55a925ed7d958a]) successfully deleted
I20260811 06:35:00.234591 28243 catalog_manager.cc:4841] TS 9084db42d92e4d99a500de07ecdb741c (127.28.239.195:42081): tablet ef9036ca44364222b5911d9474f24755 (table test-table [id=44220b37addd4c089d55a925ed7d958a]) successfully deleted
I20260811 06:35:00.342906 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 28358
I20260811 06:35:00.343016 28242 catalog_manager.cc:4841] TS 29e6eba07d584b298acefddfc15f40ba (127.28.239.193:43215): tablet ef9036ca44364222b5911d9474f24755 (table test-table [id=44220b37addd4c089d55a925ed7d958a]) successfully deleted
I20260811 06:35:00.374509 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 28547
I20260811 06:35:00.382958 28960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:00.375369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53706 (local address 127.28.239.194:44831)
0811 06:35:00.375573 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:00.375579 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:00.375605 (+ 26us) server_negotiation.cc:407] Connection header received
0811 06:35:00.375822 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:00.375828 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:00.375980 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:00.376088 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:00.376518 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:00.377810 (+ 1292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:00.379089 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:00.379844 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:00.379980 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:00.381076 (+ 1096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:35:00.381088 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:35:00.381111 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:35:00.382048 (+ 937us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:35:00.382119 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:35:00.382147 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:35:00.382267 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 06:35:00.382414 (+ 147us) server_negotiation.cc:299] Negotiation successful
0811 06:35:00.382527 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 06:35:00.383811 28721 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:35:00.501475 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 28547
I20260811 06:35:00.531150 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 28735
I20260811 06:35:00.538823 28961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:00.532047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59464 (local address 127.28.239.195:42081)
0811 06:35:00.532254 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:00.532261 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:00.532282 (+ 21us) server_negotiation.cc:407] Connection header received
0811 06:35:00.532467 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:00.532473 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:00.532579 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:00.532695 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:00.533136 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:00.533976 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:00.535520 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:00.535952 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:00.536115 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:00.537268 (+ 1153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:35:00.537280 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:35:00.537298 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:35:00.538025 (+ 727us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:35:00.538075 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:35:00.538097 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:35:00.538238 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 06:35:00.538377 (+ 139us) server_negotiation.cc:299] Negotiation successful
0811 06:35:00.538488 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 06:35:00.539589 28927 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:35:00.631759 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 28735
I20260811 06:35:00.657883 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 28222
I20260811 06:35:00.659061 28312 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:35:00.771039 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 28222
2026-08-11T06:35:00Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (3393 ms)
[----------] 6 tests from EncryptionPolicyTest (25043 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786429755927508-29631-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:35:00 dist-test-slave-gkw7 krb5kdc[29016](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:35:00 dist-test-slave-gkw7 krb5kdc[29016](info): set up 2 sockets
Aug 11 06:35:00 dist-test-slave-gkw7 krb5kdc[29016](info): commencing operation
krb5kdc: starting...
W20260811 06:35:02.156412 29631 mini_kdc.cc:121] Time spent starting KDC: real 1.328s user 0.000s sys 0.052s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786429755927508-29631-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786429755927508-29631-0/krb5kdc/test-user.keytab.
Aug 11 06:35:02 dist-test-slave-gkw7 krb5kdc[29016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430102, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:35:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:35:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786429755927508-29631-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786429755927508-29631-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 06:35:02.509166 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786429755927508-29631-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786429755927508-29631-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786429755927508-29631-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46609
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786429755927508-29631-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46609
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:35:02.869282 29037 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:35:02.869692 29037 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:35:02.869865 29037 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:35:02.877164 29037 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:35:02.877326 29037 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:35:02.877444 29037 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:35:02.877593 29037 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:35:02.877668 29037 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:35:02.887881 29037 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786429755927508-29631-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786429755927508-29631-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46609
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786429755927508-29631-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46609
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786429755927508-29631-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:35:02.889335 29037 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:35:02.890823 29037 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:35:02.903364 29046 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:35:02.903453 29044 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:35:02.903491 29042 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:35:02.904677 29037 server_base.cc:1034] running on GCE node
I20260811 06:35:02.905714 29037 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:35:02.907378 29037 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:35:02.908676 29037 hybrid_clock.cc:648] HybridClock initialized: now 1786430102908598 us; error 51 us; skew 500 ppm
I20260811 06:35:02.909169 29037 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:35:02 dist-test-slave-gkw7 krb5kdc[29016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430102, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:35:02.913020 29037 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 06:35:02.914727 29037 webserver.cc:460] Webserver started at http://127.0.0.1:46263/ using document root <none> and password file <none>
I20260811 06:35:02.915413 29037 fs_manager.cc:359] Metadata directory not provided
I20260811 06:35:02.915520 29037 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:35:02.915796 29037 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:35:02.919076 29037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786429755927508-29631-0/minicluster-data/master-0/data/instance:
uuid: "908ccd0f1bdf49988e9c03a1accf1070"
format_stamp: "Formatted at 2026-08-11 06:35:02 on dist-test-slave-gkw7"
server_key: "767a8834d02ed6e275e15055c86550a6"
server_key_iv: "b73ad73f74f69dce1453f72215d452fa"
server_key_version: "encryptionkey@0"
I20260811 06:35:02.919884 29037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786429755927508-29631-0/minicluster-data/master-0/wal/instance:
uuid: "908ccd0f1bdf49988e9c03a1accf1070"
format_stamp: "Formatted at 2026-08-11 06:35:02 on dist-test-slave-gkw7"
server_key: "767a8834d02ed6e275e15055c86550a6"
server_key_iv: "b73ad73f74f69dce1453f72215d452fa"
server_key_version: "encryptionkey@0"
I20260811 06:35:02.925384 29037 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260811 06:35:02.929134 29057 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:35:02.930617 29037 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:35:02.930774 29037 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786429755927508-29631-0/minicluster-data/master-0/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786429755927508-29631-0/minicluster-data/master-0/wal
uuid: "908ccd0f1bdf49988e9c03a1accf1070"
format_stamp: "Formatted at 2026-08-11 06:35:02 on dist-test-slave-gkw7"
server_key: "767a8834d02ed6e275e15055c86550a6"
server_key_iv: "b73ad73f74f69dce1453f72215d452fa"
server_key_version: "encryptionkey@0"
I20260811 06:35:02.930984 29037 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786429755927508-29631-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786429755927508-29631-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786429755927508-29631-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:35:02.959448 29037 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:35:02.964056 29037 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:35:02.964347 29037 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:35:02.984886 29037 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46609
I20260811 06:35:02.984913 29142 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46609 every 8 connection(s)
I20260811 06:35:02.986433 29037 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
I20260811 06:35:02.990557 29145 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:35:02.994412 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 29037
I20260811 06:35:02.994648 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786429755927508-29631-0/minicluster-data/master-0/wal/instance
I20260811 06:35:02.995340 29631 external_mini_cluster.cc:1428] Setting key 5c50a21efa04fcc85fcb7a7fe24f7a8c
I20260811 06:35:03.002305 29145 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 908ccd0f1bdf49988e9c03a1accf1070: Bootstrap starting.
I20260811 06:35:03.006168 29145 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 908ccd0f1bdf49988e9c03a1accf1070: Neither blocks nor log segments found. Creating new log.
I20260811 06:35:03.007401 29145 log.cc:824] T 00000000000000000000000000000000 P 908ccd0f1bdf49988e9c03a1accf1070: Log is configured to *not* fsync() on all Append() calls
I20260811 06:35:03.011143 29145 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 908ccd0f1bdf49988e9c03a1accf1070: No bootstrap required, opened a new log
Aug 11 06:35:03 dist-test-slave-gkw7 krb5kdc[29016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430102, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 06:35:03.022754 29145 raft_consensus.cc:348] T 00000000000000000000000000000000 P 908ccd0f1bdf49988e9c03a1accf1070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "908ccd0f1bdf49988e9c03a1accf1070" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46609 } }
I20260811 06:35:03.023319 29145 raft_consensus.cc:374] T 00000000000000000000000000000000 P 908ccd0f1bdf49988e9c03a1accf1070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:35:03.023336 29149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:02.997349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60369 (local address 127.0.0.1:46609)
0811 06:35:02.998207 (+ 858us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:02.998220 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:02.998262 (+ 42us) server_negotiation.cc:407] Connection header received
0811 06:35:03.000632 (+ 2370us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:03.000659 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:03.001103 (+ 444us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:03.001441 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:03.003403 (+ 1962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:03.004783 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:03.006780 (+ 1997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:03.007479 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:03.007671 (+ 192us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:03.012994 (+ 5323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:35:03.013005 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:35:03.013033 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:35:03.013095 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:35:03.017867 (+ 4772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:35:03.018906 (+ 1039us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:35:03.018915 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:35:03.018926 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:35:03.019167 (+ 241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:35:03.019696 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:35:03.019701 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:35:03.019705 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:35:03.020539 (+ 834us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:35:03.020923 (+ 384us) server_negotiation.cc:1035] Waiting for connection context
0811 06:35:03.021370 (+ 447us) server_negotiation.cc:299] Negotiation successful
0811 06:35:03.021803 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":638,"thread_start_us":422,"threads_started":1}
I20260811 06:35:03.023465 29145 raft_consensus.cc:729] T 00000000000000000000000000000000 P 908ccd0f1bdf49988e9c03a1accf1070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 908ccd0f1bdf49988e9c03a1accf1070, State: Initialized, Role: FOLLOWER
I20260811 06:35:03.024081 29145 consensus_queue.cc:260] T 00000000000000000000000000000000 P 908ccd0f1bdf49988e9c03a1accf1070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "908ccd0f1bdf49988e9c03a1accf1070" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46609 } }
I20260811 06:35:03.024367 29145 raft_consensus.cc:388] T 00000000000000000000000000000000 P 908ccd0f1bdf49988e9c03a1accf1070 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:35:03.024523 29145 raft_consensus.cc:482] T 00000000000000000000000000000000 P 908ccd0f1bdf49988e9c03a1accf1070 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:35:03.024713 29145 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 908ccd0f1bdf49988e9c03a1accf1070 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:35:03.027500 29145 raft_consensus.cc:504] T 00000000000000000000000000000000 P 908ccd0f1bdf49988e9c03a1accf1070 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "908ccd0f1bdf49988e9c03a1accf1070" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46609 } }
I20260811 06:35:03.028019 29631 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 06:35:03.028095 29145 leader_election.cc:302] T 00000000000000000000000000000000 P 908ccd0f1bdf49988e9c03a1accf1070 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 908ccd0f1bdf49988e9c03a1accf1070; no voters:
I20260811 06:35:03.028772 29145 leader_election.cc:288] T 00000000000000000000000000000000 P 908ccd0f1bdf49988e9c03a1accf1070 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:35:03.028910 29154 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 908ccd0f1bdf49988e9c03a1accf1070 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:35:03.030303 29154 raft_consensus.cc:686] T 00000000000000000000000000000000 P 908ccd0f1bdf49988e9c03a1accf1070 [term 1 LEADER]: Becoming Leader. State: Replica: 908ccd0f1bdf49988e9c03a1accf1070, State: Running, Role: LEADER
I20260811 06:35:03.030956 29154 consensus_queue.cc:237] T 00000000000000000000000000000000 P 908ccd0f1bdf49988e9c03a1accf1070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "908ccd0f1bdf49988e9c03a1accf1070" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46609 } }
I20260811 06:35:03.031267 29145 sys_catalog.cc:564] T 00000000000000000000000000000000 P 908ccd0f1bdf49988e9c03a1accf1070 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:35:03.034282 29158 sys_catalog.cc:455] T 00000000000000000000000000000000 P 908ccd0f1bdf49988e9c03a1accf1070 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 908ccd0f1bdf49988e9c03a1accf1070. Latest consensus state: current_term: 1 leader_uuid: "908ccd0f1bdf49988e9c03a1accf1070" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "908ccd0f1bdf49988e9c03a1accf1070" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46609 } } }
I20260811 06:35:03.034192 29155 sys_catalog.cc:455] T 00000000000000000000000000000000 P 908ccd0f1bdf49988e9c03a1accf1070 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "908ccd0f1bdf49988e9c03a1accf1070" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "908ccd0f1bdf49988e9c03a1accf1070" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46609 } } }
I20260811 06:35:03.034711 29158 sys_catalog.cc:458] T 00000000000000000000000000000000 P 908ccd0f1bdf49988e9c03a1accf1070 [sys.catalog]: This master's current role is: LEADER
I20260811 06:35:03.034711 29155 sys_catalog.cc:458] T 00000000000000000000000000000000 P 908ccd0f1bdf49988e9c03a1accf1070 [sys.catalog]: This master's current role is: LEADER
I20260811 06:35:03.035676 29169 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:35:03.052704 29169 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:35:03.065807 29149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:03.041213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38237 (local address 127.0.0.1:46609)
0811 06:35:03.041443 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:03.041453 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:03.042268 (+ 815us) server_negotiation.cc:407] Connection header received
0811 06:35:03.042600 (+ 332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:03.042607 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:03.042709 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:03.042808 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:03.045651 (+ 2843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:03.047282 (+ 1631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:03.049876 (+ 2594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:03.050920 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:03.051055 (+ 135us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:03.055624 (+ 4569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:35:03.055638 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:35:03.055646 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:35:03.055703 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:35:03.060755 (+ 5052us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:35:03.062206 (+ 1451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:35:03.062215 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:35:03.062222 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:35:03.062442 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:35:03.063178 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:35:03.063185 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:35:03.063191 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:35:03.063720 (+ 529us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:35:03.064076 (+ 356us) server_negotiation.cc:1035] Waiting for connection context
0811 06:35:03.064983 (+ 907us) server_negotiation.cc:299] Negotiation successful
0811 06:35:03.065249 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 06:35:03.072178 29169 catalog_manager.cc:1347] Generated new cluster ID: e69271b7dc3b40f09304801122d57365
I20260811 06:35:03.072355 29169 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:35:03.101249 29169 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:35:03.102720 29169 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:35:03.114481 29169 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 908ccd0f1bdf49988e9c03a1accf1070: Generated new TSK 0
I20260811 06:35:03.115698 29169 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:35:03.131472 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 29037
I20260811 06:35:03.133355 29135 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:35:03.231247 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 29037
W20260811 06:35:03.258121 29014 connection.cc:535] client connection to 127.0.0.1:46609 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46609): Connection reset by peer (error 104)
2026-08-11T06:35:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2450 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786429755927508-29631-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:35:03 dist-test-slave-gkw7 krb5kdc[29195](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:35:03 dist-test-slave-gkw7 krb5kdc[29195](info): set up 2 sockets
Aug 11 06:35:03 dist-test-slave-gkw7 krb5kdc[29195](info): commencing operation
krb5kdc: starting...
W20260811 06:35:04.596657 29631 mini_kdc.cc:121] Time spent starting KDC: real 1.316s user 0.001s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786429755927508-29631-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786429755927508-29631-0/krb5kdc/test-user.keytab.
Aug 11 06:35:04 dist-test-slave-gkw7 krb5kdc[29195](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430104, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:35:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:35:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786429755927508-29631-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786429755927508-29631-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 06:35:04.937002 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786429755927508-29631-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786429755927508-29631-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786429755927508-29631-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46411
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786429755927508-29631-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46411
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:35:05.306681 29221 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:35:05.307044 29221 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:35:05.307255 29221 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:35:05.314613 29221 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:35:05.314793 29221 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:35:05.314908 29221 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:35:05.315028 29221 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:35:05.315119 29221 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:35:05.325357 29221 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786429755927508-29631-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786429755927508-29631-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46411
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786429755927508-29631-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46411
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786429755927508-29631-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:35:05.326789 29221 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:35:05.328224 29221 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:35:05.340430 29229 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:35:05.340485 29231 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:35:05.340487 29233 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:35:05.341439 29221 server_base.cc:1034] running on GCE node
I20260811 06:35:05.342260 29221 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:35:05.343734 29221 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:35:05.344946 29221 hybrid_clock.cc:648] HybridClock initialized: now 1786430105344891 us; error 56 us; skew 500 ppm
I20260811 06:35:05.345274 29221 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:35:05 dist-test-slave-gkw7 krb5kdc[29195](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430105, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:35:05.356206 29221 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 06:35:05.358014 29221 webserver.cc:460] Webserver started at http://127.0.0.1:37757/ using document root <none> and password file <none>
I20260811 06:35:05.358765 29221 fs_manager.cc:359] Metadata directory not provided
I20260811 06:35:05.358918 29221 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:35:05.359256 29221 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:35:05.362578 29221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786429755927508-29631-0/minicluster-data/master-0/data/instance:
uuid: "45c617631b6c4582b26c306780e1aebd"
format_stamp: "Formatted at 2026-08-11 06:35:05 on dist-test-slave-gkw7"
server_key: "f2cf756e94b6fcdc019a998c64e320f2"
server_key_iv: "15353aa7c786c6b39435dafa5dfd0502"
server_key_version: "encryptionkey@0"
I20260811 06:35:05.363404 29221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786429755927508-29631-0/minicluster-data/master-0/wal/instance:
uuid: "45c617631b6c4582b26c306780e1aebd"
format_stamp: "Formatted at 2026-08-11 06:35:05 on dist-test-slave-gkw7"
server_key: "f2cf756e94b6fcdc019a998c64e320f2"
server_key_iv: "15353aa7c786c6b39435dafa5dfd0502"
server_key_version: "encryptionkey@0"
I20260811 06:35:05.369341 29221 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260811 06:35:05.372936 29242 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:35:05.374395 29221 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:35:05.374589 29221 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786429755927508-29631-0/minicluster-data/master-0/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786429755927508-29631-0/minicluster-data/master-0/wal
uuid: "45c617631b6c4582b26c306780e1aebd"
format_stamp: "Formatted at 2026-08-11 06:35:05 on dist-test-slave-gkw7"
server_key: "f2cf756e94b6fcdc019a998c64e320f2"
server_key_iv: "15353aa7c786c6b39435dafa5dfd0502"
server_key_version: "encryptionkey@0"
I20260811 06:35:05.374811 29221 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786429755927508-29631-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786429755927508-29631-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786429755927508-29631-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:35:05.390394 29221 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:35:05.394783 29221 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:35:05.395090 29221 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:35:05.414608 29221 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46411
I20260811 06:35:05.414634 29323 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46411 every 8 connection(s)
I20260811 06:35:05.416114 29221 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
I20260811 06:35:05.420109 29325 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:35:05.422044 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 29221
I20260811 06:35:05.422266 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786429755927508-29631-0/minicluster-data/master-0/wal/instance
I20260811 06:35:05.422817 29631 external_mini_cluster.cc:1428] Setting key d8e55f44be9cd6f62bb0b3a64ec90ad8
I20260811 06:35:05.435168 29325 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45c617631b6c4582b26c306780e1aebd: Bootstrap starting.
I20260811 06:35:05.436982 29330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:05.424746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41313 (local address 127.0.0.1:46411)
0811 06:35:05.425659 (+ 913us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:05.425682 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:05.425728 (+ 46us) server_negotiation.cc:407] Connection header received
0811 06:35:05.427359 (+ 1631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:05.427388 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:05.427829 (+ 441us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:05.428172 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:05.428768 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:05.430152 (+ 1384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:05.432116 (+ 1964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:05.432886 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:05.433018 (+ 132us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:05.434450 (+ 1432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:35:05.434458 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:35:05.434474 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:35:05.435069 (+ 595us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:35:05.435160 (+ 91us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:35:05.435186 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:35:05.435428 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0811 06:35:05.435706 (+ 278us) server_negotiation.cc:299] Negotiation successful
0811 06:35:05.435937 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":622,"thread_start_us":396,"threads_started":1}
I20260811 06:35:05.438921 29325 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 45c617631b6c4582b26c306780e1aebd: Neither blocks nor log segments found. Creating new log.
I20260811 06:35:05.440316 29325 log.cc:824] T 00000000000000000000000000000000 P 45c617631b6c4582b26c306780e1aebd: Log is configured to *not* fsync() on all Append() calls
I20260811 06:35:05.441702 29631 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 06:35:05.443934 29325 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45c617631b6c4582b26c306780e1aebd: No bootstrap required, opened a new log
I20260811 06:35:05.454579 29330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:05.447125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55373 (local address 127.0.0.1:46411)
0811 06:35:05.447335 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:05.447347 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:05.447911 (+ 564us) server_negotiation.cc:407] Connection header received
0811 06:35:05.448056 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:05.448063 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:05.448158 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:05.448279 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:05.448832 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:05.450130 (+ 1298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:05.451499 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:05.452229 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:05.452367 (+ 138us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:05.453591 (+ 1224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:35:05.453598 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:35:05.453604 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:35:05.453627 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:35:05.453677 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:35:05.453702 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:35:05.453827 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 06:35:05.454037 (+ 210us) server_negotiation.cc:299] Negotiation successful
0811 06:35:05.454313 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 06:35:05.454684 29325 raft_consensus.cc:348] T 00000000000000000000000000000000 P 45c617631b6c4582b26c306780e1aebd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45c617631b6c4582b26c306780e1aebd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46411 } }
I20260811 06:35:05.455236 29325 raft_consensus.cc:374] T 00000000000000000000000000000000 P 45c617631b6c4582b26c306780e1aebd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:35:05.455381 29325 raft_consensus.cc:729] T 00000000000000000000000000000000 P 45c617631b6c4582b26c306780e1aebd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45c617631b6c4582b26c306780e1aebd, State: Initialized, Role: FOLLOWER
I20260811 06:35:05.456113 29325 consensus_queue.cc:260] T 00000000000000000000000000000000 P 45c617631b6c4582b26c306780e1aebd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45c617631b6c4582b26c306780e1aebd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46411 } }
I20260811 06:35:05.456395 29325 raft_consensus.cc:388] T 00000000000000000000000000000000 P 45c617631b6c4582b26c306780e1aebd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:35:05.456555 29325 raft_consensus.cc:482] T 00000000000000000000000000000000 P 45c617631b6c4582b26c306780e1aebd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:35:05.456743 29325 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 45c617631b6c4582b26c306780e1aebd [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:35:05.459198 29325 raft_consensus.cc:504] T 00000000000000000000000000000000 P 45c617631b6c4582b26c306780e1aebd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45c617631b6c4582b26c306780e1aebd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46411 } }
I20260811 06:35:05.459779 29325 leader_election.cc:302] T 00000000000000000000000000000000 P 45c617631b6c4582b26c306780e1aebd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 45c617631b6c4582b26c306780e1aebd; no voters:
I20260811 06:35:05.460400 29325 leader_election.cc:288] T 00000000000000000000000000000000 P 45c617631b6c4582b26c306780e1aebd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:35:05.460556 29343 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 45c617631b6c4582b26c306780e1aebd [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:35:05.461711 29343 raft_consensus.cc:686] T 00000000000000000000000000000000 P 45c617631b6c4582b26c306780e1aebd [term 1 LEADER]: Becoming Leader. State: Replica: 45c617631b6c4582b26c306780e1aebd, State: Running, Role: LEADER
I20260811 06:35:05.462349 29343 consensus_queue.cc:237] T 00000000000000000000000000000000 P 45c617631b6c4582b26c306780e1aebd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45c617631b6c4582b26c306780e1aebd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46411 } }
I20260811 06:35:05.462915 29325 sys_catalog.cc:564] T 00000000000000000000000000000000 P 45c617631b6c4582b26c306780e1aebd [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:35:05.465973 29351 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45c617631b6c4582b26c306780e1aebd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 45c617631b6c4582b26c306780e1aebd. Latest consensus state: current_term: 1 leader_uuid: "45c617631b6c4582b26c306780e1aebd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45c617631b6c4582b26c306780e1aebd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46411 } } }
I20260811 06:35:05.466267 29346 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45c617631b6c4582b26c306780e1aebd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "45c617631b6c4582b26c306780e1aebd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45c617631b6c4582b26c306780e1aebd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46411 } } }
I20260811 06:35:05.466472 29351 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45c617631b6c4582b26c306780e1aebd [sys.catalog]: This master's current role is: LEADER
I20260811 06:35:05.466547 29346 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45c617631b6c4582b26c306780e1aebd [sys.catalog]: This master's current role is: LEADER
I20260811 06:35:05.467538 29359 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:35:05.473589 29359 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:35:05.483834 29359 catalog_manager.cc:1347] Generated new cluster ID: 8ddce35034e24bee877fbaa9198efccd
I20260811 06:35:05.483969 29359 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:35:05.496644 29359 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:35:05.498092 29359 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:35:05.509653 29359 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 45c617631b6c4582b26c306780e1aebd: Generated new TSK 0
I20260811 06:35:05.510581 29359 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:35:05.516882 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 29221
W20260811 06:35:05.518651 29317 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:41313
W20260811 06:35:05.519457 29631 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 29221: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 06:35:05.519658 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 29221
W20260811 06:35:05.549794 29193 connection.cc:535] client connection to 127.0.0.1:46411 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46411): Connection reset by peer (error 104)
2026-08-11T06:35:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (2291 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786429755927508-29631-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:35:05 dist-test-slave-gkw7 krb5kdc[29382](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:35:05 dist-test-slave-gkw7 krb5kdc[29382](info): set up 2 sockets
Aug 11 06:35:05 dist-test-slave-gkw7 krb5kdc[29382](info): commencing operation
krb5kdc: starting...
W20260811 06:35:06.895907 29631 mini_kdc.cc:121] Time spent starting KDC: real 1.327s user 0.008s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786429755927508-29631-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786429755927508-29631-0/krb5kdc/test-user.keytab.
Aug 11 06:35:07 dist-test-slave-gkw7 krb5kdc[29382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430107, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:35:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:35:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786429755927508-29631-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786429755927508-29631-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 06:35:07.240808 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786429755927508-29631-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786429755927508-29631-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786429755927508-29631-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40747
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786429755927508-29631-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40747
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:35:07.607321 29410 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:35:07.607623 29410 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:35:07.607719 29410 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:35:07.615190 29410 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:35:07.615314 29410 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:35:07.615362 29410 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:35:07.615424 29410 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:35:07.615469 29410 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:35:07.625730 29410 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786429755927508-29631-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786429755927508-29631-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40747
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786429755927508-29631-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40747
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786429755927508-29631-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:35:07.627170 29410 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:35:07.628494 29410 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:35:07.640940 29418 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:35:07.640861 29420 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:35:07.640908 29424 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:35:07.641907 29410 server_base.cc:1034] running on GCE node
I20260811 06:35:07.642733 29410 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:35:07.644318 29410 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:35:07.645596 29410 hybrid_clock.cc:648] HybridClock initialized: now 1786430107645511 us; error 58 us; skew 500 ppm
I20260811 06:35:07.645967 29410 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:35:07 dist-test-slave-gkw7 krb5kdc[29382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430107, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:35:07.649151 29410 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 06:35:07.650703 29410 webserver.cc:460] Webserver started at http://127.0.0.1:43549/ using document root <none> and password file <none>
I20260811 06:35:07.651410 29410 fs_manager.cc:359] Metadata directory not provided
I20260811 06:35:07.651563 29410 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:35:07.651890 29410 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:35:07.655117 29410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786429755927508-29631-0/minicluster-data/master-0/data/instance:
uuid: "ade0eafca5e3493f95f5d21d6142ed56"
format_stamp: "Formatted at 2026-08-11 06:35:07 on dist-test-slave-gkw7"
server_key: "170386f008f1d009de23a550ec06751f"
server_key_iv: "7c32bf90823e63ad260f753a119e6491"
server_key_version: "encryptionkey@0"
I20260811 06:35:07.655928 29410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786429755927508-29631-0/minicluster-data/master-0/wal/instance:
uuid: "ade0eafca5e3493f95f5d21d6142ed56"
format_stamp: "Formatted at 2026-08-11 06:35:07 on dist-test-slave-gkw7"
server_key: "170386f008f1d009de23a550ec06751f"
server_key_iv: "7c32bf90823e63ad260f753a119e6491"
server_key_version: "encryptionkey@0"
I20260811 06:35:07.661774 29410 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260811 06:35:07.665485 29433 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:35:07.666960 29410 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:35:07.667102 29410 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786429755927508-29631-0/minicluster-data/master-0/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786429755927508-29631-0/minicluster-data/master-0/wal
uuid: "ade0eafca5e3493f95f5d21d6142ed56"
format_stamp: "Formatted at 2026-08-11 06:35:07 on dist-test-slave-gkw7"
server_key: "170386f008f1d009de23a550ec06751f"
server_key_iv: "7c32bf90823e63ad260f753a119e6491"
server_key_version: "encryptionkey@0"
I20260811 06:35:07.667275 29410 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786429755927508-29631-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786429755927508-29631-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786429755927508-29631-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:35:07.673142 29410 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:35:07.677254 29410 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:35:07.677515 29410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:35:07.697059 29410 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40747
I20260811 06:35:07.697108 29517 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40747 every 8 connection(s)
I20260811 06:35:07.698606 29410 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
I20260811 06:35:07.702709 29519 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:35:07.705236 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 29410
I20260811 06:35:07.705425 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786429755927508-29631-0/minicluster-data/master-0/wal/instance
I20260811 06:35:07.706037 29631 external_mini_cluster.cc:1428] Setting key 3d29acda22dbfa23f4098f7ac62c5f35
I20260811 06:35:07.716481 29523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:07.708358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34375 (local address 127.0.0.1:40747)
0811 06:35:07.709378 (+ 1020us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:07.709395 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:07.709839 (+ 444us) server_negotiation.cc:407] Connection header received
0811 06:35:07.712308 (+ 2469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:07.712345 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:07.712858 (+ 513us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:07.713351 (+ 493us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:07.714446 (+ 1095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:35:07.714455 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:35:07.714479 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:35:07.714502 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:35:07.714570 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:35:07.714606 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:35:07.714863 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0811 06:35:07.715003 (+ 140us) server_negotiation.cc:299] Negotiation successful
0811 06:35:07.715190 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":738,"thread_start_us":429,"threads_started":1}
I20260811 06:35:07.717232 29519 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ade0eafca5e3493f95f5d21d6142ed56: Bootstrap starting.
I20260811 06:35:07.720391 29631 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 06:35:07.720957 29519 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ade0eafca5e3493f95f5d21d6142ed56: Neither blocks nor log segments found. Creating new log.
I20260811 06:35:07.722117 29519 log.cc:824] T 00000000000000000000000000000000 P ade0eafca5e3493f95f5d21d6142ed56: Log is configured to *not* fsync() on all Append() calls
I20260811 06:35:07.725785 29519 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ade0eafca5e3493f95f5d21d6142ed56: No bootstrap required, opened a new log
I20260811 06:35:07.727689 29523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:07.725167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50959 (local address 127.0.0.1:40747)
0811 06:35:07.725346 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:07.725352 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:07.725749 (+ 397us) server_negotiation.cc:407] Connection header received
0811 06:35:07.725892 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:07.725897 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:07.725957 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:07.726073 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:07.727031 (+ 958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:35:07.727038 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:35:07.727042 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:35:07.727061 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:35:07.727107 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:35:07.727126 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:35:07.727252 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 06:35:07.727336 (+ 84us) server_negotiation.cc:299] Negotiation successful
0811 06:35:07.727393 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 06:35:07.734476 29519 raft_consensus.cc:348] T 00000000000000000000000000000000 P ade0eafca5e3493f95f5d21d6142ed56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ade0eafca5e3493f95f5d21d6142ed56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40747 } }
I20260811 06:35:07.734882 29519 raft_consensus.cc:374] T 00000000000000000000000000000000 P ade0eafca5e3493f95f5d21d6142ed56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:35:07.735036 29519 raft_consensus.cc:729] T 00000000000000000000000000000000 P ade0eafca5e3493f95f5d21d6142ed56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ade0eafca5e3493f95f5d21d6142ed56, State: Initialized, Role: FOLLOWER
I20260811 06:35:07.735702 29519 consensus_queue.cc:260] T 00000000000000000000000000000000 P ade0eafca5e3493f95f5d21d6142ed56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ade0eafca5e3493f95f5d21d6142ed56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40747 } }
I20260811 06:35:07.735960 29519 raft_consensus.cc:388] T 00000000000000000000000000000000 P ade0eafca5e3493f95f5d21d6142ed56 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:35:07.736117 29519 raft_consensus.cc:482] T 00000000000000000000000000000000 P ade0eafca5e3493f95f5d21d6142ed56 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:35:07.736294 29519 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ade0eafca5e3493f95f5d21d6142ed56 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:35:07.738641 29519 raft_consensus.cc:504] T 00000000000000000000000000000000 P ade0eafca5e3493f95f5d21d6142ed56 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ade0eafca5e3493f95f5d21d6142ed56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40747 } }
I20260811 06:35:07.739156 29519 leader_election.cc:302] T 00000000000000000000000000000000 P ade0eafca5e3493f95f5d21d6142ed56 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ade0eafca5e3493f95f5d21d6142ed56; no voters:
I20260811 06:35:07.739820 29519 leader_election.cc:288] T 00000000000000000000000000000000 P ade0eafca5e3493f95f5d21d6142ed56 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:35:07.739982 29532 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ade0eafca5e3493f95f5d21d6142ed56 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:35:07.740693 29532 raft_consensus.cc:686] T 00000000000000000000000000000000 P ade0eafca5e3493f95f5d21d6142ed56 [term 1 LEADER]: Becoming Leader. State: Replica: ade0eafca5e3493f95f5d21d6142ed56, State: Running, Role: LEADER
I20260811 06:35:07.741307 29532 consensus_queue.cc:237] T 00000000000000000000000000000000 P ade0eafca5e3493f95f5d21d6142ed56 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ade0eafca5e3493f95f5d21d6142ed56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40747 } }
I20260811 06:35:07.742156 29519 sys_catalog.cc:564] T 00000000000000000000000000000000 P ade0eafca5e3493f95f5d21d6142ed56 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:35:07.744660 29533 sys_catalog.cc:455] T 00000000000000000000000000000000 P ade0eafca5e3493f95f5d21d6142ed56 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ade0eafca5e3493f95f5d21d6142ed56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ade0eafca5e3493f95f5d21d6142ed56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40747 } } }
I20260811 06:35:07.745218 29533 sys_catalog.cc:458] T 00000000000000000000000000000000 P ade0eafca5e3493f95f5d21d6142ed56 [sys.catalog]: This master's current role is: LEADER
I20260811 06:35:07.745046 29534 sys_catalog.cc:455] T 00000000000000000000000000000000 P ade0eafca5e3493f95f5d21d6142ed56 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ade0eafca5e3493f95f5d21d6142ed56. Latest consensus state: current_term: 1 leader_uuid: "ade0eafca5e3493f95f5d21d6142ed56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ade0eafca5e3493f95f5d21d6142ed56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40747 } } }
I20260811 06:35:07.745561 29534 sys_catalog.cc:458] T 00000000000000000000000000000000 P ade0eafca5e3493f95f5d21d6142ed56 [sys.catalog]: This master's current role is: LEADER
I20260811 06:35:07.751319 29543 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:35:07.756790 29543 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:35:07.766244 29543 catalog_manager.cc:1347] Generated new cluster ID: e0ae5a7b3c154d4598c0e05e4df0b6ee
I20260811 06:35:07.766381 29543 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:35:07.794919 29543 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:35:07.795396 29543 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:35:07.818154 29543 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ade0eafca5e3493f95f5d21d6142ed56: Generated new TSK 0
I20260811 06:35:07.819373 29543 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:35:07.838521 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 29410
W20260811 06:35:07.839787 29510 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:34375
W20260811 06:35:07.840528 29631 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 29410: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 06:35:07.840685 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 29410
2026-08-11T06:35:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (2319 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786429755927508-29631-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:35:07 dist-test-slave-gkw7 krb5kdc[29562](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:35:07 dist-test-slave-gkw7 krb5kdc[29562](info): set up 2 sockets
Aug 11 06:35:07 dist-test-slave-gkw7 krb5kdc[29562](info): commencing operation
krb5kdc: starting...
W20260811 06:35:09.278482 29631 mini_kdc.cc:121] Time spent starting KDC: real 1.389s user 0.000s sys 0.050s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786429755927508-29631-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786429755927508-29631-0/krb5kdc/test-user.keytab.
Aug 11 06:35:09 dist-test-slave-gkw7 krb5kdc[29562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430109, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:35:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:35:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786429755927508-29631-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786429755927508-29631-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 06:35:09.625180 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786429755927508-29631-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786429755927508-29631-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786429755927508-29631-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34809
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786429755927508-29631-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34809
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:35:09.983448 29584 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:35:09.983775 29584 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:35:09.983958 29584 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:35:09.991364 29584 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:35:09.991468 29584 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:35:09.991578 29584 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:35:09.991689 29584 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:35:09.991779 29584 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:35:10.001682 29584 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786429755927508-29631-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786429755927508-29631-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34809
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786429755927508-29631-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34809
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786429755927508-29631-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:35:10.003049 29584 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:35:10.004460 29584 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:35:10.017233 29602 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:35:10.017161 29598 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:35:10.017196 29593 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:35:10.018275 29584 server_base.cc:1034] running on GCE node
I20260811 06:35:10.019196 29584 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:35:10.020656 29584 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:35:10.021871 29584 hybrid_clock.cc:648] HybridClock initialized: now 1786430110021819 us; error 47 us; skew 500 ppm
I20260811 06:35:10.022289 29584 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:35:10 dist-test-slave-gkw7 krb5kdc[29562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430110, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:35:10.025631 29584 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 06:35:10.027164 29584 webserver.cc:460] Webserver started at http://127.0.0.1:41453/ using document root <none> and password file <none>
I20260811 06:35:10.028218 29584 fs_manager.cc:359] Metadata directory not provided
I20260811 06:35:10.028384 29584 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:35:10.028731 29584 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:35:10.031792 29584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786429755927508-29631-0/minicluster-data/master-0/data/instance:
uuid: "4b484fa9eabe444fbf5eb67504624425"
format_stamp: "Formatted at 2026-08-11 06:35:10 on dist-test-slave-gkw7"
server_key: "5c5463a06749c07ab3dec6c1823d82da"
server_key_iv: "eb896a41f7dcec117b51dd62a2817450"
server_key_version: "encryptionkey@0"
I20260811 06:35:10.032539 29584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786429755927508-29631-0/minicluster-data/master-0/wal/instance:
uuid: "4b484fa9eabe444fbf5eb67504624425"
format_stamp: "Formatted at 2026-08-11 06:35:10 on dist-test-slave-gkw7"
server_key: "5c5463a06749c07ab3dec6c1823d82da"
server_key_iv: "eb896a41f7dcec117b51dd62a2817450"
server_key_version: "encryptionkey@0"
I20260811 06:35:10.037860 29584 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 06:35:10.041356 29612 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:35:10.042811 29584 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:35:10.042953 29584 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786429755927508-29631-0/minicluster-data/master-0/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786429755927508-29631-0/minicluster-data/master-0/wal
uuid: "4b484fa9eabe444fbf5eb67504624425"
format_stamp: "Formatted at 2026-08-11 06:35:10 on dist-test-slave-gkw7"
server_key: "5c5463a06749c07ab3dec6c1823d82da"
server_key_iv: "eb896a41f7dcec117b51dd62a2817450"
server_key_version: "encryptionkey@0"
I20260811 06:35:10.043123 29584 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786429755927508-29631-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786429755927508-29631-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786429755927508-29631-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:35:10.078336 29584 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:35:10.082530 29584 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:35:10.082779 29584 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:35:10.102094 29584 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34809
I20260811 06:35:10.102129 29702 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34809 every 8 connection(s)
I20260811 06:35:10.103590 29584 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
I20260811 06:35:10.107551 29704 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:35:10.113245 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 29584
I20260811 06:35:10.113477 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786429755927508-29631-0/minicluster-data/master-0/wal/instance
I20260811 06:35:10.114094 29631 external_mini_cluster.cc:1428] Setting key 767e498a4d63ea5099f4eceba817a8f0
I20260811 06:35:10.120314 29704 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b484fa9eabe444fbf5eb67504624425: Bootstrap starting.
I20260811 06:35:10.123637 29704 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4b484fa9eabe444fbf5eb67504624425: Neither blocks nor log segments found. Creating new log.
I20260811 06:35:10.124727 29704 log.cc:824] T 00000000000000000000000000000000 P 4b484fa9eabe444fbf5eb67504624425: Log is configured to *not* fsync() on all Append() calls
I20260811 06:35:10.127776 29704 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b484fa9eabe444fbf5eb67504624425: No bootstrap required, opened a new log
Aug 11 06:35:10 dist-test-slave-gkw7 krb5kdc[29562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430109, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 06:35:10.136299 29704 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4b484fa9eabe444fbf5eb67504624425 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b484fa9eabe444fbf5eb67504624425" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34809 } }
I20260811 06:35:10.136706 29704 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4b484fa9eabe444fbf5eb67504624425 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:35:10.136839 29704 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4b484fa9eabe444fbf5eb67504624425 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b484fa9eabe444fbf5eb67504624425, State: Initialized, Role: FOLLOWER
I20260811 06:35:10.137511 29704 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4b484fa9eabe444fbf5eb67504624425 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b484fa9eabe444fbf5eb67504624425" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34809 } }
I20260811 06:35:10.137903 29704 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4b484fa9eabe444fbf5eb67504624425 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:35:10.138062 29704 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4b484fa9eabe444fbf5eb67504624425 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:35:10.138250 29704 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4b484fa9eabe444fbf5eb67504624425 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:35:10.138492 29707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:10.116108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35407 (local address 127.0.0.1:34809)
0811 06:35:10.116987 (+ 879us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:10.117004 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:10.117049 (+ 45us) server_negotiation.cc:407] Connection header received
0811 06:35:10.118954 (+ 1905us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:10.118979 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:10.119477 (+ 498us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:10.119880 (+ 403us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:10.121468 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:10.122957 (+ 1489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:10.124290 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:10.125028 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:10.125232 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:10.129579 (+ 4347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:35:10.129592 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:35:10.129617 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:35:10.129657 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:35:10.133339 (+ 3682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:35:10.134225 (+ 886us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:35:10.134243 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:35:10.134251 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:35:10.134455 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:35:10.134923 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:35:10.134929 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:35:10.134935 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:35:10.135576 (+ 641us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:35:10.135893 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0811 06:35:10.136444 (+ 551us) server_negotiation.cc:299] Negotiation successful
0811 06:35:10.137137 (+ 693us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":653,"thread_start_us":404,"threads_started":1}
I20260811 06:35:10.141284 29704 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4b484fa9eabe444fbf5eb67504624425 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b484fa9eabe444fbf5eb67504624425" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34809 } }
I20260811 06:35:10.141927 29704 leader_election.cc:302] T 00000000000000000000000000000000 P 4b484fa9eabe444fbf5eb67504624425 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4b484fa9eabe444fbf5eb67504624425; no voters:
I20260811 06:35:10.142673 29704 leader_election.cc:288] T 00000000000000000000000000000000 P 4b484fa9eabe444fbf5eb67504624425 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:35:10.142740 29631 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 06:35:10.142796 29711 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4b484fa9eabe444fbf5eb67504624425 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:35:10.145283 29704 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4b484fa9eabe444fbf5eb67504624425 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:35:10.148422 29711 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4b484fa9eabe444fbf5eb67504624425 [term 1 LEADER]: Becoming Leader. State: Replica: 4b484fa9eabe444fbf5eb67504624425, State: Running, Role: LEADER
I20260811 06:35:10.149255 29711 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4b484fa9eabe444fbf5eb67504624425 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b484fa9eabe444fbf5eb67504624425" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34809 } }
I20260811 06:35:10.152921 29712 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b484fa9eabe444fbf5eb67504624425 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4b484fa9eabe444fbf5eb67504624425" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b484fa9eabe444fbf5eb67504624425" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34809 } } }
I20260811 06:35:10.153417 29712 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b484fa9eabe444fbf5eb67504624425 [sys.catalog]: This master's current role is: LEADER
I20260811 06:35:10.154076 29713 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b484fa9eabe444fbf5eb67504624425 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4b484fa9eabe444fbf5eb67504624425. Latest consensus state: current_term: 1 leader_uuid: "4b484fa9eabe444fbf5eb67504624425" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b484fa9eabe444fbf5eb67504624425" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34809 } } }
I20260811 06:35:10.154389 29713 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b484fa9eabe444fbf5eb67504624425 [sys.catalog]: This master's current role is: LEADER
I20260811 06:35:10.156680 29744 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:35:10.164947 29744 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:35:10.169090 29707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:10.148344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51735 (local address 127.0.0.1:34809)
0811 06:35:10.148544 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:10.148549 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:10.149455 (+ 906us) server_negotiation.cc:407] Connection header received
0811 06:35:10.149703 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:10.149709 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:10.149810 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:10.149930 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:10.152036 (+ 2106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:10.153159 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:10.155471 (+ 2312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:10.156479 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:10.156649 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:10.160694 (+ 4045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:35:10.160702 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:35:10.160712 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:35:10.160764 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:35:10.164772 (+ 4008us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:35:10.165740 (+ 968us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:35:10.165746 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:35:10.165751 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:35:10.165895 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:35:10.166431 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:35:10.166438 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:35:10.166442 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:35:10.166864 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:35:10.167118 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0811 06:35:10.168474 (+ 1356us) server_negotiation.cc:299] Negotiation successful
0811 06:35:10.168711 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 06:35:10.177623 29744 catalog_manager.cc:1347] Generated new cluster ID: 74fb5dec8e7d4de491f6f91b581f4e09
I20260811 06:35:10.177752 29744 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:35:10.190125 29744 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:35:10.191560 29744 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:35:10.200641 29744 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4b484fa9eabe444fbf5eb67504624425: Generated new TSK 0
I20260811 06:35:10.201449 29744 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:35:10.209825 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 29584
I20260811 06:35:10.211987 29696 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:35:10.304292 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 29584
2026-08-11T06:35:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2459 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786429755927508-29631-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:35:10 dist-test-slave-gkw7 krb5kdc[29760](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:35:10 dist-test-slave-gkw7 krb5kdc[29760](info): set up 2 sockets
Aug 11 06:35:10 dist-test-slave-gkw7 krb5kdc[29760](info): commencing operation
krb5kdc: starting...
W20260811 06:35:11.739106 29631 mini_kdc.cc:121] Time spent starting KDC: real 1.391s user 0.002s sys 0.049s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786429755927508-29631-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786429755927508-29631-0/krb5kdc/test-user.keytab.
Aug 11 06:35:11 dist-test-slave-gkw7 krb5kdc[29760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430111, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:35:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:35:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786429755927508-29631-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786429755927508-29631-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 06:35:12.084666 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786429755927508-29631-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786429755927508-29631-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786429755927508-29631-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40309
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786429755927508-29631-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40309
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:35:12.438300 29797 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:35:12.438611 29797 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:35:12.438841 29797 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:35:12.446282 29797 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:35:12.446436 29797 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:35:12.446552 29797 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:35:12.446635 29797 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:35:12.446708 29797 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:35:12.456977 29797 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786429755927508-29631-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786429755927508-29631-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40309
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786429755927508-29631-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40309
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786429755927508-29631-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:35:12.458416 29797 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:35:12.459825 29797 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:35:12.472249 29811 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:35:12.472343 29808 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:35:12.472689 29806 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:35:12.473043 29797 server_base.cc:1034] running on GCE node
I20260811 06:35:12.474021 29797 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:35:12.475447 29797 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:35:12.476701 29797 hybrid_clock.cc:648] HybridClock initialized: now 1786430112476646 us; error 40 us; skew 500 ppm
I20260811 06:35:12.477063 29797 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:35:12 dist-test-slave-gkw7 krb5kdc[29760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430112, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:35:12.480304 29797 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 06:35:12.481909 29797 webserver.cc:460] Webserver started at http://127.0.0.1:43355/ using document root <none> and password file <none>
I20260811 06:35:12.482604 29797 fs_manager.cc:359] Metadata directory not provided
I20260811 06:35:12.482759 29797 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:35:12.483076 29797 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:35:12.486248 29797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786429755927508-29631-0/minicluster-data/master-0/data/instance:
uuid: "49c8753c8e154f869cde37175aa465cd"
format_stamp: "Formatted at 2026-08-11 06:35:12 on dist-test-slave-gkw7"
server_key: "3d8edcd581a7fc23a12a31ec92d67642"
server_key_iv: "6c40c22a7974ee5626e9408f3a31f26e"
server_key_version: "encryptionkey@0"
I20260811 06:35:12.487056 29797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786429755927508-29631-0/minicluster-data/master-0/wal/instance:
uuid: "49c8753c8e154f869cde37175aa465cd"
format_stamp: "Formatted at 2026-08-11 06:35:12 on dist-test-slave-gkw7"
server_key: "3d8edcd581a7fc23a12a31ec92d67642"
server_key_iv: "6c40c22a7974ee5626e9408f3a31f26e"
server_key_version: "encryptionkey@0"
I20260811 06:35:12.492726 29797 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 06:35:12.496423 29824 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:35:12.497889 29797 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:35:12.498078 29797 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786429755927508-29631-0/minicluster-data/master-0/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786429755927508-29631-0/minicluster-data/master-0/wal
uuid: "49c8753c8e154f869cde37175aa465cd"
format_stamp: "Formatted at 2026-08-11 06:35:12 on dist-test-slave-gkw7"
server_key: "3d8edcd581a7fc23a12a31ec92d67642"
server_key_iv: "6c40c22a7974ee5626e9408f3a31f26e"
server_key_version: "encryptionkey@0"
I20260811 06:35:12.498292 29797 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786429755927508-29631-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786429755927508-29631-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786429755927508-29631-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:35:12.522800 29797 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:35:12.527079 29797 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:35:12.527395 29797 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:35:12.547209 29797 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40309
I20260811 06:35:12.547231 29911 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40309 every 8 connection(s)
I20260811 06:35:12.548799 29797 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
I20260811 06:35:12.552671 29912 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:35:12.558476 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 29797
I20260811 06:35:12.558668 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786429755927508-29631-0/minicluster-data/master-0/wal/instance
I20260811 06:35:12.559194 29631 external_mini_cluster.cc:1428] Setting key 17a4f6ffab8dd6098b001bc6b8fc5c68
I20260811 06:35:12.564509 29912 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49c8753c8e154f869cde37175aa465cd: Bootstrap starting.
I20260811 06:35:12.568406 29912 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 49c8753c8e154f869cde37175aa465cd: Neither blocks nor log segments found. Creating new log.
I20260811 06:35:12.569674 29912 log.cc:824] T 00000000000000000000000000000000 P 49c8753c8e154f869cde37175aa465cd: Log is configured to *not* fsync() on all Append() calls
I20260811 06:35:12.573169 29912 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49c8753c8e154f869cde37175aa465cd: No bootstrap required, opened a new log
I20260811 06:35:12.573736 29916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:12.561137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49883 (local address 127.0.0.1:40309)
0811 06:35:12.562083 (+ 946us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:12.562105 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:12.562148 (+ 43us) server_negotiation.cc:407] Connection header received
0811 06:35:12.563864 (+ 1716us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:12.563887 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:12.564313 (+ 426us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:12.564686 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:12.565315 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:12.566798 (+ 1483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:12.568732 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:12.569461 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:12.569673 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:12.571012 (+ 1339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:35:12.571025 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:35:12.571044 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:35:12.571675 (+ 631us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:35:12.571757 (+ 82us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:35:12.571782 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:35:12.571872 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 06:35:12.572289 (+ 417us) server_negotiation.cc:299] Negotiation successful
0811 06:35:12.572615 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":699,"thread_start_us":436,"threads_started":1}
I20260811 06:35:12.577590 29631 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 06:35:12.583479 29912 raft_consensus.cc:348] T 00000000000000000000000000000000 P 49c8753c8e154f869cde37175aa465cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49c8753c8e154f869cde37175aa465cd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40309 } }
I20260811 06:35:12.584050 29912 raft_consensus.cc:374] T 00000000000000000000000000000000 P 49c8753c8e154f869cde37175aa465cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:35:12.584149 29912 raft_consensus.cc:729] T 00000000000000000000000000000000 P 49c8753c8e154f869cde37175aa465cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49c8753c8e154f869cde37175aa465cd, State: Initialized, Role: FOLLOWER
I20260811 06:35:12.584946 29912 consensus_queue.cc:260] T 00000000000000000000000000000000 P 49c8753c8e154f869cde37175aa465cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49c8753c8e154f869cde37175aa465cd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40309 } }
I20260811 06:35:12.585224 29912 raft_consensus.cc:388] T 00000000000000000000000000000000 P 49c8753c8e154f869cde37175aa465cd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:35:12.585404 29912 raft_consensus.cc:482] T 00000000000000000000000000000000 P 49c8753c8e154f869cde37175aa465cd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:35:12.585618 29912 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 49c8753c8e154f869cde37175aa465cd [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:35:12.588871 29912 raft_consensus.cc:504] T 00000000000000000000000000000000 P 49c8753c8e154f869cde37175aa465cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49c8753c8e154f869cde37175aa465cd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40309 } }
I20260811 06:35:12.589555 29912 leader_election.cc:302] T 00000000000000000000000000000000 P 49c8753c8e154f869cde37175aa465cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 49c8753c8e154f869cde37175aa465cd; no voters:
I20260811 06:35:12.590333 29912 leader_election.cc:288] T 00000000000000000000000000000000 P 49c8753c8e154f869cde37175aa465cd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:35:12.590461 29929 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 49c8753c8e154f869cde37175aa465cd [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:35:12.590533 29916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:12.583176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43533 (local address 127.0.0.1:40309)
0811 06:35:12.583431 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:12.583438 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:12.583816 (+ 378us) server_negotiation.cc:407] Connection header received
0811 06:35:12.584056 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:12.584067 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:12.584192 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:12.584299 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:12.584908 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:12.585889 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:12.587774 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:12.588228 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:12.588324 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:12.589650 (+ 1326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:35:12.589655 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:35:12.589658 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:35:12.589675 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:35:12.589735 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:35:12.589757 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:35:12.589850 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 06:35:12.590003 (+ 153us) server_negotiation.cc:299] Negotiation successful
0811 06:35:12.590099 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 06:35:12.592300 29929 raft_consensus.cc:686] T 00000000000000000000000000000000 P 49c8753c8e154f869cde37175aa465cd [term 1 LEADER]: Becoming Leader. State: Replica: 49c8753c8e154f869cde37175aa465cd, State: Running, Role: LEADER
I20260811 06:35:12.592932 29929 consensus_queue.cc:237] T 00000000000000000000000000000000 P 49c8753c8e154f869cde37175aa465cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49c8753c8e154f869cde37175aa465cd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40309 } }
I20260811 06:35:12.593135 29912 sys_catalog.cc:564] T 00000000000000000000000000000000 P 49c8753c8e154f869cde37175aa465cd [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:35:12.596275 29930 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49c8753c8e154f869cde37175aa465cd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "49c8753c8e154f869cde37175aa465cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49c8753c8e154f869cde37175aa465cd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40309 } } }
I20260811 06:35:12.596714 29930 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49c8753c8e154f869cde37175aa465cd [sys.catalog]: This master's current role is: LEADER
I20260811 06:35:12.597211 29931 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49c8753c8e154f869cde37175aa465cd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 49c8753c8e154f869cde37175aa465cd. Latest consensus state: current_term: 1 leader_uuid: "49c8753c8e154f869cde37175aa465cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49c8753c8e154f869cde37175aa465cd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40309 } } }
I20260811 06:35:12.597476 29931 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49c8753c8e154f869cde37175aa465cd [sys.catalog]: This master's current role is: LEADER
I20260811 06:35:12.603171 29939 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:35:12.608741 29939 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:35:12.618415 29939 catalog_manager.cc:1347] Generated new cluster ID: 48838b6bfd954ff2b4f139d4a66d80b4
I20260811 06:35:12.618593 29939 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:35:12.653514 29939 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:35:12.655145 29939 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:35:12.667529 29939 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 49c8753c8e154f869cde37175aa465cd: Generated new TSK 0
I20260811 06:35:12.668443 29939 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:35:12.688879 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 29797
W20260811 06:35:12.690140 29906 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:49883
W20260811 06:35:12.690899 29631 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 29797: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 06:35:12.691059 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 29797
2026-08-11T06:35:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (2387 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786429755927508-29631-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:35:12 dist-test-slave-gkw7 krb5kdc[29963](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:35:12 dist-test-slave-gkw7 krb5kdc[29963](info): set up 2 sockets
Aug 11 06:35:12 dist-test-slave-gkw7 krb5kdc[29963](info): commencing operation
krb5kdc: starting...
W20260811 06:35:14.098155 29631 mini_kdc.cc:121] Time spent starting KDC: real 1.363s user 0.003s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786429755927508-29631-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786429755927508-29631-0/krb5kdc/test-user.keytab.
Aug 11 06:35:14 dist-test-slave-gkw7 krb5kdc[29963](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430114, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:35:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:35:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786429755927508-29631-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786429755927508-29631-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 06:35:14.453171 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786429755927508-29631-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786429755927508-29631-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786429755927508-29631-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39507
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786429755927508-29631-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39507
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:35:14.812768 29983 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:35:14.813078 29983 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:35:14.813242 29983 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:35:14.820680 29983 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:35:14.820804 29983 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:35:14.820890 29983 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:35:14.820997 29983 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:35:14.821077 29983 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:35:14.831287 29983 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786429755927508-29631-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786429755927508-29631-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39507
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786429755927508-29631-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39507
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786429755927508-29631-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:35:14.832779 29983 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:35:14.834228 29983 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:35:14.846724 29991 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:35:14.846794 29994 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:35:14.846827 29996 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:35:14.847977 29983 server_base.cc:1034] running on GCE node
I20260811 06:35:14.848829 29983 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:35:14.850474 29983 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:35:14.851698 29983 hybrid_clock.cc:648] HybridClock initialized: now 1786430114851639 us; error 54 us; skew 500 ppm
I20260811 06:35:14.852095 29983 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:35:14 dist-test-slave-gkw7 krb5kdc[29963](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430114, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:35:14.855720 29983 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 06:35:14.857285 29983 webserver.cc:460] Webserver started at http://127.0.0.1:39573/ using document root <none> and password file <none>
I20260811 06:35:14.858052 29983 fs_manager.cc:359] Metadata directory not provided
I20260811 06:35:14.858215 29983 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:35:14.858547 29983 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:35:14.861750 29983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786429755927508-29631-0/minicluster-data/master-0/data/instance:
uuid: "69184ba6772e4d5c92b78ee9d0b53ea5"
format_stamp: "Formatted at 2026-08-11 06:35:14 on dist-test-slave-gkw7"
server_key: "9626136b6b1c9293a609d5dbdc5829f8"
server_key_iv: "b79429d0b0cadc79ef225b0e09337577"
server_key_version: "encryptionkey@0"
I20260811 06:35:14.862605 29983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786429755927508-29631-0/minicluster-data/master-0/wal/instance:
uuid: "69184ba6772e4d5c92b78ee9d0b53ea5"
format_stamp: "Formatted at 2026-08-11 06:35:14 on dist-test-slave-gkw7"
server_key: "9626136b6b1c9293a609d5dbdc5829f8"
server_key_iv: "b79429d0b0cadc79ef225b0e09337577"
server_key_version: "encryptionkey@0"
I20260811 06:35:14.868203 29983 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 06:35:14.872301 30006 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:35:14.873816 29983 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:35:14.873972 29983 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786429755927508-29631-0/minicluster-data/master-0/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786429755927508-29631-0/minicluster-data/master-0/wal
uuid: "69184ba6772e4d5c92b78ee9d0b53ea5"
format_stamp: "Formatted at 2026-08-11 06:35:14 on dist-test-slave-gkw7"
server_key: "9626136b6b1c9293a609d5dbdc5829f8"
server_key_iv: "b79429d0b0cadc79ef225b0e09337577"
server_key_version: "encryptionkey@0"
I20260811 06:35:14.874152 29983 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786429755927508-29631-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786429755927508-29631-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786429755927508-29631-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:35:14.880160 29983 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:35:14.884364 29983 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:35:14.884624 29983 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:35:14.904673 29983 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39507
I20260811 06:35:14.904731 30095 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39507 every 8 connection(s)
I20260811 06:35:14.906220 29983 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
I20260811 06:35:14.907810 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 29983
I20260811 06:35:14.908001 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786429755927508-29631-0/minicluster-data/master-0/wal/instance
I20260811 06:35:14.908496 29631 external_mini_cluster.cc:1428] Setting key bc0c39414136b8b98c23fff1f67203d2
I20260811 06:35:14.910970 30096 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:35:14.918141 30103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:14.910493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55109 (local address 127.0.0.1:39507)
0811 06:35:14.911424 (+ 931us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:14.911441 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:14.911496 (+ 55us) server_negotiation.cc:407] Connection header received
0811 06:35:14.913933 (+ 2437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:14.913963 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:14.914488 (+ 525us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:14.914968 (+ 480us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:14.916013 (+ 1045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:35:14.916021 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:35:14.916038 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:35:14.916054 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:35:14.916160 (+ 106us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:35:14.916191 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:35:14.916424 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0811 06:35:14.916574 (+ 150us) server_negotiation.cc:299] Negotiation successful
0811 06:35:14.916780 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":660,"thread_start_us":357,"threads_started":1}
I20260811 06:35:14.921957 29631 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 06:35:14.925629 30096 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69184ba6772e4d5c92b78ee9d0b53ea5: Bootstrap starting.
I20260811 06:35:14.930092 30096 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 69184ba6772e4d5c92b78ee9d0b53ea5: Neither blocks nor log segments found. Creating new log.
I20260811 06:35:14.930066 30103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:14.927386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52617 (local address 127.0.0.1:39507)
0811 06:35:14.927646 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:14.927652 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:14.928086 (+ 434us) server_negotiation.cc:407] Connection header received
0811 06:35:14.928279 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:14.928283 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:14.928333 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:14.928413 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:14.929384 (+ 971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:35:14.929389 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:35:14.929392 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:35:14.929406 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:35:14.929446 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:35:14.929461 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:35:14.929593 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 06:35:14.929708 (+ 115us) server_negotiation.cc:299] Negotiation successful
0811 06:35:14.929758 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 06:35:14.931574 30096 log.cc:824] T 00000000000000000000000000000000 P 69184ba6772e4d5c92b78ee9d0b53ea5: Log is configured to *not* fsync() on all Append() calls
I20260811 06:35:14.934456 30096 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69184ba6772e4d5c92b78ee9d0b53ea5: No bootstrap required, opened a new log
I20260811 06:35:14.942207 30096 raft_consensus.cc:348] T 00000000000000000000000000000000 P 69184ba6772e4d5c92b78ee9d0b53ea5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69184ba6772e4d5c92b78ee9d0b53ea5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39507 } }
I20260811 06:35:14.942817 30096 raft_consensus.cc:374] T 00000000000000000000000000000000 P 69184ba6772e4d5c92b78ee9d0b53ea5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:35:14.942947 30096 raft_consensus.cc:729] T 00000000000000000000000000000000 P 69184ba6772e4d5c92b78ee9d0b53ea5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69184ba6772e4d5c92b78ee9d0b53ea5, State: Initialized, Role: FOLLOWER
I20260811 06:35:14.943612 30096 consensus_queue.cc:260] T 00000000000000000000000000000000 P 69184ba6772e4d5c92b78ee9d0b53ea5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69184ba6772e4d5c92b78ee9d0b53ea5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39507 } }
I20260811 06:35:14.943887 30096 raft_consensus.cc:388] T 00000000000000000000000000000000 P 69184ba6772e4d5c92b78ee9d0b53ea5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:35:14.944052 30096 raft_consensus.cc:482] T 00000000000000000000000000000000 P 69184ba6772e4d5c92b78ee9d0b53ea5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:35:14.944293 30096 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 69184ba6772e4d5c92b78ee9d0b53ea5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:35:14.947068 30096 raft_consensus.cc:504] T 00000000000000000000000000000000 P 69184ba6772e4d5c92b78ee9d0b53ea5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69184ba6772e4d5c92b78ee9d0b53ea5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39507 } }
I20260811 06:35:14.947651 30096 leader_election.cc:302] T 00000000000000000000000000000000 P 69184ba6772e4d5c92b78ee9d0b53ea5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 69184ba6772e4d5c92b78ee9d0b53ea5; no voters:
I20260811 06:35:14.948276 30096 leader_election.cc:288] T 00000000000000000000000000000000 P 69184ba6772e4d5c92b78ee9d0b53ea5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:35:14.948413 30111 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 69184ba6772e4d5c92b78ee9d0b53ea5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:35:14.949321 30111 raft_consensus.cc:686] T 00000000000000000000000000000000 P 69184ba6772e4d5c92b78ee9d0b53ea5 [term 1 LEADER]: Becoming Leader. State: Replica: 69184ba6772e4d5c92b78ee9d0b53ea5, State: Running, Role: LEADER
I20260811 06:35:14.949908 30111 consensus_queue.cc:237] T 00000000000000000000000000000000 P 69184ba6772e4d5c92b78ee9d0b53ea5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69184ba6772e4d5c92b78ee9d0b53ea5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39507 } }
I20260811 06:35:14.950793 30096 sys_catalog.cc:564] T 00000000000000000000000000000000 P 69184ba6772e4d5c92b78ee9d0b53ea5 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:35:14.953156 30113 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69184ba6772e4d5c92b78ee9d0b53ea5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 69184ba6772e4d5c92b78ee9d0b53ea5. Latest consensus state: current_term: 1 leader_uuid: "69184ba6772e4d5c92b78ee9d0b53ea5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69184ba6772e4d5c92b78ee9d0b53ea5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39507 } } }
I20260811 06:35:14.953809 30113 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69184ba6772e4d5c92b78ee9d0b53ea5 [sys.catalog]: This master's current role is: LEADER
I20260811 06:35:14.952942 30112 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69184ba6772e4d5c92b78ee9d0b53ea5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "69184ba6772e4d5c92b78ee9d0b53ea5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69184ba6772e4d5c92b78ee9d0b53ea5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39507 } } }
I20260811 06:35:14.956645 30112 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69184ba6772e4d5c92b78ee9d0b53ea5 [sys.catalog]: This master's current role is: LEADER
I20260811 06:35:14.956843 30124 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:35:14.962690 30124 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:35:14.973315 30124 catalog_manager.cc:1347] Generated new cluster ID: 690866c0dfe841cb95045712c4107035
I20260811 06:35:14.973469 30124 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:35:14.988679 30124 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:35:14.989128 30124 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:35:14.996989 30124 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 69184ba6772e4d5c92b78ee9d0b53ea5: Generated new TSK 0
I20260811 06:35:14.998108 30124 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:35:15.015859 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 29983
W20260811 06:35:15.017154 30088 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:55109
W20260811 06:35:15.017937 29631 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 29983: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 06:35:15.018114 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 29983
2026-08-11T06:35:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (2332 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786429755927508-29631-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:35:15 dist-test-slave-gkw7 krb5kdc[30147](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:35:15 dist-test-slave-gkw7 krb5kdc[30147](info): set up 2 sockets
Aug 11 06:35:15 dist-test-slave-gkw7 krb5kdc[30147](info): commencing operation
krb5kdc: starting...
W20260811 06:35:16.430613 29631 mini_kdc.cc:121] Time spent starting KDC: real 1.361s user 0.003s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786429755927508-29631-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786429755927508-29631-0/krb5kdc/test-user.keytab.
Aug 11 06:35:16 dist-test-slave-gkw7 krb5kdc[30147](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430116, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:35:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:35:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786429755927508-29631-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786429755927508-29631-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 06:35:16.786489 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786429755927508-29631-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786429755927508-29631-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786429755927508-29631-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36461
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786429755927508-29631-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36461
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:35:17.151456 30174 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:35:17.151814 30174 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:35:17.151988 30174 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:35:17.159327 30174 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:35:17.159504 30174 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:35:17.159608 30174 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:35:17.159718 30174 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:35:17.159801 30174 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:35:17.169919 30174 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786429755927508-29631-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786429755927508-29631-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36461
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786429755927508-29631-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36461
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786429755927508-29631-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:35:17.171372 30174 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:35:17.172850 30174 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:35:17.185137 30182 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:35:17.185230 30185 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:35:17.185249 30183 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:35:17.186082 30174 server_base.cc:1034] running on GCE node
I20260811 06:35:17.186969 30174 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:35:17.188478 30174 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:35:17.189710 30174 hybrid_clock.cc:648] HybridClock initialized: now 1786430117189661 us; error 44 us; skew 500 ppm
I20260811 06:35:17.190092 30174 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:35:17 dist-test-slave-gkw7 krb5kdc[30147](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430117, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:35:17.198712 30174 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 06:35:17.200372 30174 webserver.cc:460] Webserver started at http://127.0.0.1:41055/ using document root <none> and password file <none>
I20260811 06:35:17.201087 30174 fs_manager.cc:359] Metadata directory not provided
I20260811 06:35:17.201195 30174 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:35:17.201463 30174 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:35:17.204684 30174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786429755927508-29631-0/minicluster-data/master-0/data/instance:
uuid: "138bc9552fba4e6da665077d10f14f3c"
format_stamp: "Formatted at 2026-08-11 06:35:17 on dist-test-slave-gkw7"
server_key: "db06814b6a8b60c692270566988a7b1c"
server_key_iv: "1f0563678d19440a90a04656625dd12e"
server_key_version: "encryptionkey@0"
I20260811 06:35:17.205448 30174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786429755927508-29631-0/minicluster-data/master-0/wal/instance:
uuid: "138bc9552fba4e6da665077d10f14f3c"
format_stamp: "Formatted at 2026-08-11 06:35:17 on dist-test-slave-gkw7"
server_key: "db06814b6a8b60c692270566988a7b1c"
server_key_iv: "1f0563678d19440a90a04656625dd12e"
server_key_version: "encryptionkey@0"
I20260811 06:35:17.210976 30174 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 06:35:17.214879 30196 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:35:17.216683 30174 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.002s
I20260811 06:35:17.216868 30174 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786429755927508-29631-0/minicluster-data/master-0/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786429755927508-29631-0/minicluster-data/master-0/wal
uuid: "138bc9552fba4e6da665077d10f14f3c"
format_stamp: "Formatted at 2026-08-11 06:35:17 on dist-test-slave-gkw7"
server_key: "db06814b6a8b60c692270566988a7b1c"
server_key_iv: "1f0563678d19440a90a04656625dd12e"
server_key_version: "encryptionkey@0"
I20260811 06:35:17.217078 30174 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786429755927508-29631-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786429755927508-29631-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786429755927508-29631-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:35:17.241725 30174 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:35:17.246702 30174 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:35:17.247040 30174 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:35:17.266881 30174 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36461
I20260811 06:35:17.266920 30282 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36461 every 8 connection(s)
I20260811 06:35:17.268453 30174 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
I20260811 06:35:17.272488 30283 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:35:17.273414 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 30174
I20260811 06:35:17.273638 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786429755927508-29631-0/minicluster-data/master-0/wal/instance
I20260811 06:35:17.274252 29631 external_mini_cluster.cc:1428] Setting key f12cab6140a14aecb80d2f4cb2a05136
I20260811 06:35:17.290165 30283 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 138bc9552fba4e6da665077d10f14f3c: Bootstrap starting.
Aug 11 06:35:17 dist-test-slave-gkw7 krb5kdc[30147](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430116, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 06:35:17.294054 30283 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 138bc9552fba4e6da665077d10f14f3c: Neither blocks nor log segments found. Creating new log.
I20260811 06:35:17.295742 30283 log.cc:824] T 00000000000000000000000000000000 P 138bc9552fba4e6da665077d10f14f3c: Log is configured to *not* fsync() on all Append() calls
I20260811 06:35:17.299136 30283 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 138bc9552fba4e6da665077d10f14f3c: No bootstrap required, opened a new log
I20260811 06:35:17.302198 30291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:17.276641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.125:41669 (local address 127.0.0.1:36461)
0811 06:35:17.280048 (+ 3407us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:17.280070 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:17.280133 (+ 63us) server_negotiation.cc:407] Connection header received
0811 06:35:17.282582 (+ 2449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:17.282631 (+ 49us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:17.283235 (+ 604us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:17.283688 (+ 453us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:17.285264 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:17.286774 (+ 1510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:17.288083 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:17.288794 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:17.288994 (+ 200us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:17.293758 (+ 4764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:35:17.293767 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:35:17.293783 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:35:17.293826 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:35:17.297598 (+ 3772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:35:17.298452 (+ 854us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:35:17.298461 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:35:17.298470 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:35:17.298669 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:35:17.299174 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:35:17.299179 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:35:17.299182 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:35:17.299790 (+ 608us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:35:17.300154 (+ 364us) server_negotiation.cc:1035] Waiting for connection context
0811 06:35:17.300637 (+ 483us) server_negotiation.cc:299] Negotiation successful
0811 06:35:17.301003 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3151,"thread_start_us":2886,"threads_started":1}
I20260811 06:35:17.306329 29631 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 06:35:17.308230 30283 raft_consensus.cc:348] T 00000000000000000000000000000000 P 138bc9552fba4e6da665077d10f14f3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "138bc9552fba4e6da665077d10f14f3c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36461 } }
I20260811 06:35:17.308662 30283 raft_consensus.cc:374] T 00000000000000000000000000000000 P 138bc9552fba4e6da665077d10f14f3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:35:17.308801 30283 raft_consensus.cc:729] T 00000000000000000000000000000000 P 138bc9552fba4e6da665077d10f14f3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 138bc9552fba4e6da665077d10f14f3c, State: Initialized, Role: FOLLOWER
I20260811 06:35:17.309432 30283 consensus_queue.cc:260] T 00000000000000000000000000000000 P 138bc9552fba4e6da665077d10f14f3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "138bc9552fba4e6da665077d10f14f3c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36461 } }
I20260811 06:35:17.309785 30283 raft_consensus.cc:388] T 00000000000000000000000000000000 P 138bc9552fba4e6da665077d10f14f3c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:35:17.309967 30283 raft_consensus.cc:482] T 00000000000000000000000000000000 P 138bc9552fba4e6da665077d10f14f3c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:35:17.310159 30283 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 138bc9552fba4e6da665077d10f14f3c [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:35:17.313215 30283 raft_consensus.cc:504] T 00000000000000000000000000000000 P 138bc9552fba4e6da665077d10f14f3c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "138bc9552fba4e6da665077d10f14f3c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36461 } }
I20260811 06:35:17.313830 30283 leader_election.cc:302] T 00000000000000000000000000000000 P 138bc9552fba4e6da665077d10f14f3c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 138bc9552fba4e6da665077d10f14f3c; no voters:
I20260811 06:35:17.314584 30283 leader_election.cc:288] T 00000000000000000000000000000000 P 138bc9552fba4e6da665077d10f14f3c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:35:17.314670 30300 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 138bc9552fba4e6da665077d10f14f3c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:35:17.316340 30300 raft_consensus.cc:686] T 00000000000000000000000000000000 P 138bc9552fba4e6da665077d10f14f3c [term 1 LEADER]: Becoming Leader. State: Replica: 138bc9552fba4e6da665077d10f14f3c, State: Running, Role: LEADER
I20260811 06:35:17.316969 30300 consensus_queue.cc:237] T 00000000000000000000000000000000 P 138bc9552fba4e6da665077d10f14f3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "138bc9552fba4e6da665077d10f14f3c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36461 } }
I20260811 06:35:17.317224 30283 sys_catalog.cc:564] T 00000000000000000000000000000000 P 138bc9552fba4e6da665077d10f14f3c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:35:17.327234 30301 sys_catalog.cc:455] T 00000000000000000000000000000000 P 138bc9552fba4e6da665077d10f14f3c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "138bc9552fba4e6da665077d10f14f3c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "138bc9552fba4e6da665077d10f14f3c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36461 } } }
I20260811 06:35:17.327951 30301 sys_catalog.cc:458] T 00000000000000000000000000000000 P 138bc9552fba4e6da665077d10f14f3c [sys.catalog]: This master's current role is: LEADER
I20260811 06:35:17.328400 30302 sys_catalog.cc:455] T 00000000000000000000000000000000 P 138bc9552fba4e6da665077d10f14f3c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 138bc9552fba4e6da665077d10f14f3c. Latest consensus state: current_term: 1 leader_uuid: "138bc9552fba4e6da665077d10f14f3c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "138bc9552fba4e6da665077d10f14f3c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36461 } } }
I20260811 06:35:17.328729 30302 sys_catalog.cc:458] T 00000000000000000000000000000000 P 138bc9552fba4e6da665077d10f14f3c [sys.catalog]: This master's current role is: LEADER
I20260811 06:35:17.331619 30310 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:35:17.333936 30291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:17.311627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.125:52451 (local address 127.0.0.1:36461)
0811 06:35:17.311831 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:17.311839 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:17.312205 (+ 366us) server_negotiation.cc:407] Connection header received
0811 06:35:17.312325 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:17.312331 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:17.312417 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:17.312535 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:17.314161 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:17.315574 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:17.317514 (+ 1940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:17.318241 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:17.318386 (+ 145us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:17.323512 (+ 5126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:35:17.323525 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:35:17.323533 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:35:17.323603 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:35:17.328492 (+ 4889us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:35:17.329961 (+ 1469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:35:17.329967 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:35:17.329971 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:35:17.330147 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:35:17.330964 (+ 817us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:35:17.330972 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:35:17.330978 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:35:17.331841 (+ 863us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:35:17.332216 (+ 375us) server_negotiation.cc:1035] Waiting for connection context
0811 06:35:17.332763 (+ 547us) server_negotiation.cc:299] Negotiation successful
0811 06:35:17.333367 (+ 604us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 06:35:17.338884 30310 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:35:17.348522 30310 catalog_manager.cc:1347] Generated new cluster ID: 35c5a889e6df4e3c85956e89215fe196
I20260811 06:35:17.348712 30310 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:35:17.392838 30310 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:35:17.394886 30310 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:35:17.402827 30310 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 138bc9552fba4e6da665077d10f14f3c: Generated new TSK 0
I20260811 06:35:17.403914 30310 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:35:17.422734 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 30174
I20260811 06:35:17.424397 30276 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:35:17.520846 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 30174
W20260811 06:35:17.549019 30143 connection.cc:535] client connection to 127.0.0.1:36461 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36461): Connection reset by peer (error 104)
2026-08-11T06:35:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2500 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786429755927508-29631-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:35:17 dist-test-slave-gkw7 krb5kdc[30333](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:35:17 dist-test-slave-gkw7 krb5kdc[30333](info): set up 2 sockets
Aug 11 06:35:17 dist-test-slave-gkw7 krb5kdc[30333](info): commencing operation
krb5kdc: starting...
W20260811 06:35:18.894822 29631 mini_kdc.cc:121] Time spent starting KDC: real 1.326s user 0.000s sys 0.050s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786429755927508-29631-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786429755927508-29631-0/krb5kdc/test-user.keytab.
Aug 11 06:35:19 dist-test-slave-gkw7 krb5kdc[30333](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430119, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:35:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:35:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786429755927508-29631-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786429755927508-29631-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 06:35:19.249825 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786429755927508-29631-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786429755927508-29631-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786429755927508-29631-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37579
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786429755927508-29631-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37579
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:35:19.619330 30358 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:35:19.619647 30358 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:35:19.619743 30358 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:35:19.627125 30358 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:35:19.627249 30358 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:35:19.627307 30358 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:35:19.627373 30358 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:35:19.627419 30358 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:35:19.637614 30358 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786429755927508-29631-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786429755927508-29631-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37579
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786429755927508-29631-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37579
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786429755927508-29631-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:35:19.638983 30358 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:35:19.640405 30358 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:35:19.653818 30367 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:35:19.653898 30368 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:35:19.653877 30370 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:35:19.655054 30358 server_base.cc:1034] running on GCE node
I20260811 06:35:19.655875 30358 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:35:19.657356 30358 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:35:19.658577 30358 hybrid_clock.cc:648] HybridClock initialized: now 1786430119658542 us; error 61 us; skew 500 ppm
I20260811 06:35:19.658911 30358 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:35:19 dist-test-slave-gkw7 krb5kdc[30333](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430119, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:35:19.662072 30358 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 06:35:19.663676 30358 webserver.cc:460] Webserver started at http://127.0.0.1:32897/ using document root <none> and password file <none>
I20260811 06:35:19.664386 30358 fs_manager.cc:359] Metadata directory not provided
I20260811 06:35:19.664520 30358 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:35:19.664839 30358 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:35:19.668011 30358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786429755927508-29631-0/minicluster-data/master-0/data/instance:
uuid: "2f42b92bab1545728af8e6753ec41fc9"
format_stamp: "Formatted at 2026-08-11 06:35:19 on dist-test-slave-gkw7"
server_key: "a18f65f799532e686eb9577d7f7198fc"
server_key_iv: "72f09bf3eba73b5bc7797427cb65daf2"
server_key_version: "encryptionkey@0"
I20260811 06:35:19.668812 30358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786429755927508-29631-0/minicluster-data/master-0/wal/instance:
uuid: "2f42b92bab1545728af8e6753ec41fc9"
format_stamp: "Formatted at 2026-08-11 06:35:19 on dist-test-slave-gkw7"
server_key: "a18f65f799532e686eb9577d7f7198fc"
server_key_iv: "72f09bf3eba73b5bc7797427cb65daf2"
server_key_version: "encryptionkey@0"
I20260811 06:35:19.674146 30358 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260811 06:35:19.677687 30378 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:35:19.678989 30358 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 06:35:19.679191 30358 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786429755927508-29631-0/minicluster-data/master-0/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786429755927508-29631-0/minicluster-data/master-0/wal
uuid: "2f42b92bab1545728af8e6753ec41fc9"
format_stamp: "Formatted at 2026-08-11 06:35:19 on dist-test-slave-gkw7"
server_key: "a18f65f799532e686eb9577d7f7198fc"
server_key_iv: "72f09bf3eba73b5bc7797427cb65daf2"
server_key_version: "encryptionkey@0"
I20260811 06:35:19.679380 30358 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786429755927508-29631-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786429755927508-29631-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786429755927508-29631-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:35:19.701297 30358 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:35:19.705457 30358 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:35:19.705786 30358 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:35:19.724872 30358 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37579
I20260811 06:35:19.724902 30465 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37579 every 8 connection(s)
I20260811 06:35:19.726298 30358 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
I20260811 06:35:19.727707 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 30358
I20260811 06:35:19.727895 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786429755927508-29631-0/minicluster-data/master-0/wal/instance
I20260811 06:35:19.728524 29631 external_mini_cluster.cc:1428] Setting key 8ba54fddb379044244937d57555bb2d6
I20260811 06:35:19.731060 30466 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:35:19.742486 30471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:19.730481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.125:47719 (local address 127.0.0.1:37579)
0811 06:35:19.731328 (+ 847us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:19.731341 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:19.731386 (+ 45us) server_negotiation.cc:407] Connection header received
0811 06:35:19.733055 (+ 1669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:19.733078 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:19.733486 (+ 408us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:19.733921 (+ 435us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:19.734532 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:19.735910 (+ 1378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:19.737818 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:19.738517 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:19.738661 (+ 144us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:19.740019 (+ 1358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:35:19.740027 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:35:19.740043 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:35:19.740637 (+ 594us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:35:19.740713 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:35:19.740739 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:35:19.740864 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 06:35:19.741219 (+ 355us) server_negotiation.cc:299] Negotiation successful
0811 06:35:19.741425 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":636,"thread_start_us":360,"threads_started":1}
I20260811 06:35:19.743948 30466 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f42b92bab1545728af8e6753ec41fc9: Bootstrap starting.
I20260811 06:35:19.746492 29631 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 06:35:19.747418 30466 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2f42b92bab1545728af8e6753ec41fc9: Neither blocks nor log segments found. Creating new log.
I20260811 06:35:19.748446 30466 log.cc:824] T 00000000000000000000000000000000 P 2f42b92bab1545728af8e6753ec41fc9: Log is configured to *not* fsync() on all Append() calls
I20260811 06:35:19.751991 30466 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f42b92bab1545728af8e6753ec41fc9: No bootstrap required, opened a new log
I20260811 06:35:19.758229 30471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:19.751110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.125:39103 (local address 127.0.0.1:37579)
0811 06:35:19.751295 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:19.751303 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:19.751808 (+ 505us) server_negotiation.cc:407] Connection header received
0811 06:35:19.752031 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:19.752036 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:19.752124 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:19.752220 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:19.752778 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:19.753667 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:19.754994 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:19.755494 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:19.755629 (+ 135us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:19.757195 (+ 1566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:35:19.757202 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:35:19.757205 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:35:19.757227 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:35:19.757276 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:35:19.757458 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:35:19.757686 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0811 06:35:19.757887 (+ 201us) server_negotiation.cc:299] Negotiation successful
0811 06:35:19.757944 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 06:35:19.760779 30466 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2f42b92bab1545728af8e6753ec41fc9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f42b92bab1545728af8e6753ec41fc9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37579 } }
I20260811 06:35:19.761189 30466 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2f42b92bab1545728af8e6753ec41fc9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:35:19.761312 30466 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2f42b92bab1545728af8e6753ec41fc9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f42b92bab1545728af8e6753ec41fc9, State: Initialized, Role: FOLLOWER
I20260811 06:35:19.761966 30466 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2f42b92bab1545728af8e6753ec41fc9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f42b92bab1545728af8e6753ec41fc9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37579 } }
I20260811 06:35:19.762223 30466 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2f42b92bab1545728af8e6753ec41fc9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:35:19.762372 30466 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2f42b92bab1545728af8e6753ec41fc9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:35:19.762555 30466 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2f42b92bab1545728af8e6753ec41fc9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:35:19.764950 30466 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2f42b92bab1545728af8e6753ec41fc9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f42b92bab1545728af8e6753ec41fc9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37579 } }
I20260811 06:35:19.765470 30466 leader_election.cc:302] T 00000000000000000000000000000000 P 2f42b92bab1545728af8e6753ec41fc9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2f42b92bab1545728af8e6753ec41fc9; no voters:
I20260811 06:35:19.766261 30466 leader_election.cc:288] T 00000000000000000000000000000000 P 2f42b92bab1545728af8e6753ec41fc9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:35:19.766398 30482 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2f42b92bab1545728af8e6753ec41fc9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:35:19.767059 30482 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2f42b92bab1545728af8e6753ec41fc9 [term 1 LEADER]: Becoming Leader. State: Replica: 2f42b92bab1545728af8e6753ec41fc9, State: Running, Role: LEADER
I20260811 06:35:19.767616 30482 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2f42b92bab1545728af8e6753ec41fc9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f42b92bab1545728af8e6753ec41fc9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37579 } }
I20260811 06:35:19.769413 30482 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f42b92bab1545728af8e6753ec41fc9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2f42b92bab1545728af8e6753ec41fc9. Latest consensus state: current_term: 1 leader_uuid: "2f42b92bab1545728af8e6753ec41fc9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f42b92bab1545728af8e6753ec41fc9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37579 } } }
I20260811 06:35:19.769730 30482 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f42b92bab1545728af8e6753ec41fc9 [sys.catalog]: This master's current role is: LEADER
I20260811 06:35:19.770521 30488 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f42b92bab1545728af8e6753ec41fc9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2f42b92bab1545728af8e6753ec41fc9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f42b92bab1545728af8e6753ec41fc9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37579 } } }
I20260811 06:35:19.771008 30488 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f42b92bab1545728af8e6753ec41fc9 [sys.catalog]: This master's current role is: LEADER
I20260811 06:35:19.774417 30466 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2f42b92bab1545728af8e6753ec41fc9 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:35:19.774653 30493 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:35:19.780365 30493 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:35:19.791141 30493 catalog_manager.cc:1347] Generated new cluster ID: 2cce8fe2fe4d49e98a61eaf36dad4f52
I20260811 06:35:19.791282 30493 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:35:19.815346 30493 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:35:19.816744 30493 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:35:19.826349 30493 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2f42b92bab1545728af8e6753ec41fc9: Generated new TSK 0
I20260811 06:35:19.827239 30493 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:35:19.844592 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 30358
W20260811 06:35:19.846017 30454 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.125:47719
W20260811 06:35:19.846901 29631 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 30358: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 06:35:19.847079 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 30358
W20260811 06:35:19.871397 30331 connection.cc:535] client connection to 127.0.0.1:37579 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37579): Connection reset by peer (error 104)
2026-08-11T06:35:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2322 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786429755927508-29631-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:35:19 dist-test-slave-gkw7 krb5kdc[30525](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:35:19 dist-test-slave-gkw7 krb5kdc[30525](info): set up 2 sockets
Aug 11 06:35:19 dist-test-slave-gkw7 krb5kdc[30525](info): commencing operation
krb5kdc: starting...
W20260811 06:35:21.231696 29631 mini_kdc.cc:121] Time spent starting KDC: real 1.341s user 0.000s sys 0.051s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786429755927508-29631-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786429755927508-29631-0/krb5kdc/test-user.keytab.
Aug 11 06:35:21 dist-test-slave-gkw7 krb5kdc[30525](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430121, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:35:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:35:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786429755927508-29631-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786429755927508-29631-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 06:35:21.582648 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786429755927508-29631-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786429755927508-29631-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786429755927508-29631-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44147
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786429755927508-29631-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44147
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:35:21.960984 30543 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:35:21.961319 30543 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:35:21.961417 30543 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:35:21.968921 30543 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:35:21.969053 30543 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:35:21.969101 30543 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:35:21.969172 30543 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:35:21.969209 30543 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:35:21.979424 30543 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786429755927508-29631-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786429755927508-29631-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44147
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786429755927508-29631-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44147
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786429755927508-29631-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:35:21.980760 30543 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:35:21.982138 30543 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:35:21.994469 30552 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:35:21.994571 30548 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:35:21.994639 30549 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:35:21.995374 30543 server_base.cc:1034] running on GCE node
I20260811 06:35:21.996260 30543 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:35:21.997831 30543 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:35:21.999084 30543 hybrid_clock.cc:648] HybridClock initialized: now 1786430121999036 us; error 54 us; skew 500 ppm
I20260811 06:35:21.999513 30543 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:35:22 dist-test-slave-gkw7 krb5kdc[30525](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430122, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:35:22.002947 30543 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 06:35:22.004431 30543 webserver.cc:460] Webserver started at http://127.0.0.1:45147/ using document root <none> and password file <none>
I20260811 06:35:22.005149 30543 fs_manager.cc:359] Metadata directory not provided
I20260811 06:35:22.005308 30543 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:35:22.005684 30543 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:35:22.008781 30543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786429755927508-29631-0/minicluster-data/master-0/data/instance:
uuid: "8355730877bf4eb79105622b969fb716"
format_stamp: "Formatted at 2026-08-11 06:35:22 on dist-test-slave-gkw7"
server_key: "2ce446e80ad67d29c035b88325e44892"
server_key_iv: "2d0166e035c57837c1bdbd2068be90a2"
server_key_version: "encryptionkey@0"
I20260811 06:35:22.009682 30543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786429755927508-29631-0/minicluster-data/master-0/wal/instance:
uuid: "8355730877bf4eb79105622b969fb716"
format_stamp: "Formatted at 2026-08-11 06:35:22 on dist-test-slave-gkw7"
server_key: "2ce446e80ad67d29c035b88325e44892"
server_key_iv: "2d0166e035c57837c1bdbd2068be90a2"
server_key_version: "encryptionkey@0"
I20260811 06:35:22.015219 30543 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260811 06:35:22.019017 30561 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:35:22.020567 30543 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 06:35:22.020818 30543 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786429755927508-29631-0/minicluster-data/master-0/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786429755927508-29631-0/minicluster-data/master-0/wal
uuid: "8355730877bf4eb79105622b969fb716"
format_stamp: "Formatted at 2026-08-11 06:35:22 on dist-test-slave-gkw7"
server_key: "2ce446e80ad67d29c035b88325e44892"
server_key_iv: "2d0166e035c57837c1bdbd2068be90a2"
server_key_version: "encryptionkey@0"
I20260811 06:35:22.021064 30543 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786429755927508-29631-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786429755927508-29631-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786429755927508-29631-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:35:22.027516 30543 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:35:22.032145 30543 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:35:22.032498 30543 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:35:22.054275 30543 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44147
I20260811 06:35:22.054374 30658 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44147 every 8 connection(s)
I20260811 06:35:22.055910 30543 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
I20260811 06:35:22.059159 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 30543
I20260811 06:35:22.059386 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786429755927508-29631-0/minicluster-data/master-0/wal/instance
I20260811 06:35:22.060014 29631 external_mini_cluster.cc:1428] Setting key 06ce6cc220fc5703ea1f92a90fce62b8
I20260811 06:35:22.060745 30659 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:35:22.068284 30665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:22.062056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.125:40947 (local address 127.0.0.1:44147)
0811 06:35:22.062878 (+ 822us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:22.062891 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:22.062932 (+ 41us) server_negotiation.cc:407] Connection header received
0811 06:35:22.064500 (+ 1568us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:22.064522 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:22.064893 (+ 371us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:22.065232 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:22.066399 (+ 1167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:35:22.066405 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:35:22.066418 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:35:22.066432 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:35:22.066522 (+ 90us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:35:22.066542 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:35:22.066746 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0811 06:35:22.066851 (+ 105us) server_negotiation.cc:299] Negotiation successful
0811 06:35:22.067060 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":597,"thread_start_us":353,"threads_started":1}
I20260811 06:35:22.072292 29631 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 06:35:22.077311 30659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8355730877bf4eb79105622b969fb716: Bootstrap starting.
I20260811 06:35:22.081431 30665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:22.078710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.125:53187 (local address 127.0.0.1:44147)
0811 06:35:22.078939 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:22.078947 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:22.079321 (+ 374us) server_negotiation.cc:407] Connection header received
0811 06:35:22.079528 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:22.079532 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:22.079593 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:22.079683 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:22.080740 (+ 1057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:35:22.080744 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:35:22.080747 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:35:22.080766 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:35:22.080805 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:35:22.080820 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:35:22.080913 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 06:35:22.081067 (+ 154us) server_negotiation.cc:299] Negotiation successful
0811 06:35:22.081119 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260811 06:35:22.082070 30659 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8355730877bf4eb79105622b969fb716: Neither blocks nor log segments found. Creating new log.
I20260811 06:35:22.083397 30659 log.cc:824] T 00000000000000000000000000000000 P 8355730877bf4eb79105622b969fb716: Log is configured to *not* fsync() on all Append() calls
I20260811 06:35:22.086467 30659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8355730877bf4eb79105622b969fb716: No bootstrap required, opened a new log
I20260811 06:35:22.094723 30659 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8355730877bf4eb79105622b969fb716 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8355730877bf4eb79105622b969fb716" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44147 } }
I20260811 06:35:22.095156 30659 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8355730877bf4eb79105622b969fb716 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:35:22.095283 30659 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8355730877bf4eb79105622b969fb716 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8355730877bf4eb79105622b969fb716, State: Initialized, Role: FOLLOWER
I20260811 06:35:22.096032 30659 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8355730877bf4eb79105622b969fb716 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8355730877bf4eb79105622b969fb716" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44147 } }
I20260811 06:35:22.096354 30659 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8355730877bf4eb79105622b969fb716 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:35:22.096534 30659 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8355730877bf4eb79105622b969fb716 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:35:22.096765 30659 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8355730877bf4eb79105622b969fb716 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:35:22.099303 30659 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8355730877bf4eb79105622b969fb716 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8355730877bf4eb79105622b969fb716" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44147 } }
I20260811 06:35:22.099879 30659 leader_election.cc:302] T 00000000000000000000000000000000 P 8355730877bf4eb79105622b969fb716 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8355730877bf4eb79105622b969fb716; no voters:
I20260811 06:35:22.100605 30659 leader_election.cc:288] T 00000000000000000000000000000000 P 8355730877bf4eb79105622b969fb716 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:35:22.100780 30675 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8355730877bf4eb79105622b969fb716 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:35:22.102037 30675 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8355730877bf4eb79105622b969fb716 [term 1 LEADER]: Becoming Leader. State: Replica: 8355730877bf4eb79105622b969fb716, State: Running, Role: LEADER
I20260811 06:35:22.102623 30675 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8355730877bf4eb79105622b969fb716 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8355730877bf4eb79105622b969fb716" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44147 } }
I20260811 06:35:22.103055 30659 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8355730877bf4eb79105622b969fb716 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:35:22.105969 30677 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8355730877bf4eb79105622b969fb716 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8355730877bf4eb79105622b969fb716. Latest consensus state: current_term: 1 leader_uuid: "8355730877bf4eb79105622b969fb716" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8355730877bf4eb79105622b969fb716" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44147 } } }
I20260811 06:35:22.105880 30676 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8355730877bf4eb79105622b969fb716 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8355730877bf4eb79105622b969fb716" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8355730877bf4eb79105622b969fb716" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44147 } } }
I20260811 06:35:22.106493 30677 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8355730877bf4eb79105622b969fb716 [sys.catalog]: This master's current role is: LEADER
I20260811 06:35:22.106498 30676 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8355730877bf4eb79105622b969fb716 [sys.catalog]: This master's current role is: LEADER
I20260811 06:35:22.110198 30683 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:35:22.115899 30683 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:35:22.126012 30683 catalog_manager.cc:1347] Generated new cluster ID: 2d17eedd351644cb84f8919a356734d1
I20260811 06:35:22.126140 30683 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:35:22.142458 30683 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:35:22.142990 30683 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:35:22.151114 30683 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8355730877bf4eb79105622b969fb716: Generated new TSK 0
I20260811 06:35:22.152267 30683 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:35:22.165817 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 30543
W20260811 06:35:22.167196 30647 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.125:40947
W20260811 06:35:22.168056 29631 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 30543: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 06:35:22.168244 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 30543
2026-08-11T06:35:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (2327 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786429755927508-29631-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:35:22 dist-test-slave-gkw7 krb5kdc[30707](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:35:22 dist-test-slave-gkw7 krb5kdc[30707](info): set up 2 sockets
Aug 11 06:35:22 dist-test-slave-gkw7 krb5kdc[30707](info): commencing operation
krb5kdc: starting...
W20260811 06:35:23.545936 29631 mini_kdc.cc:121] Time spent starting KDC: real 1.327s user 0.005s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786429755927508-29631-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786429755927508-29631-0/krb5kdc/test-user.keytab.
Aug 11 06:35:23 dist-test-slave-gkw7 krb5kdc[30707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430123, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:35:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:35:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786429755927508-29631-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786429755927508-29631-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 06:35:23.905457 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786429755927508-29631-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786429755927508-29631-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786429755927508-29631-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43309
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786429755927508-29631-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43309
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:35:24.259939 30740 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:35:24.260253 30740 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:35:24.260398 30740 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:35:24.267799 30740 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:35:24.267923 30740 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:35:24.268023 30740 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:35:24.268132 30740 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:35:24.268198 30740 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:35:24.278344 30740 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786429755927508-29631-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786429755927508-29631-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43309
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786429755927508-29631-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43309
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786429755927508-29631-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:35:24.279728 30740 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:35:24.281128 30740 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:35:24.293426 30755 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:35:24.293354 30759 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:35:24.293380 30756 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:35:24.294160 30740 server_base.cc:1034] running on GCE node
I20260811 06:35:24.294968 30740 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:35:24.296526 30740 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:35:24.297780 30740 hybrid_clock.cc:648] HybridClock initialized: now 1786430124297732 us; error 55 us; skew 500 ppm
I20260811 06:35:24.298161 30740 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:35:24 dist-test-slave-gkw7 krb5kdc[30707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430124, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:35:24.301702 30740 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 06:35:24.303369 30740 webserver.cc:460] Webserver started at http://127.0.0.1:40349/ using document root <none> and password file <none>
I20260811 06:35:24.304155 30740 fs_manager.cc:359] Metadata directory not provided
I20260811 06:35:24.304322 30740 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:35:24.304677 30740 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:35:24.308218 30740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786429755927508-29631-0/minicluster-data/master-0/data/instance:
uuid: "e86f93f0ba0e45a693e7d377f403500c"
format_stamp: "Formatted at 2026-08-11 06:35:24 on dist-test-slave-gkw7"
server_key: "9d76298dafe09e2fcba04c64506cb092"
server_key_iv: "f70c05c9287751ec3f715a630ddc5b87"
server_key_version: "encryptionkey@0"
I20260811 06:35:24.309181 30740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786429755927508-29631-0/minicluster-data/master-0/wal/instance:
uuid: "e86f93f0ba0e45a693e7d377f403500c"
format_stamp: "Formatted at 2026-08-11 06:35:24 on dist-test-slave-gkw7"
server_key: "9d76298dafe09e2fcba04c64506cb092"
server_key_iv: "f70c05c9287751ec3f715a630ddc5b87"
server_key_version: "encryptionkey@0"
I20260811 06:35:24.315065 30740 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 06:35:24.319047 30769 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:35:24.320511 30740 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 06:35:24.320737 30740 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786429755927508-29631-0/minicluster-data/master-0/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786429755927508-29631-0/minicluster-data/master-0/wal
uuid: "e86f93f0ba0e45a693e7d377f403500c"
format_stamp: "Formatted at 2026-08-11 06:35:24 on dist-test-slave-gkw7"
server_key: "9d76298dafe09e2fcba04c64506cb092"
server_key_iv: "f70c05c9287751ec3f715a630ddc5b87"
server_key_version: "encryptionkey@0"
I20260811 06:35:24.320961 30740 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786429755927508-29631-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786429755927508-29631-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786429755927508-29631-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:35:24.349715 30740 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:35:24.354982 30740 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:35:24.355332 30740 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:35:24.376677 30740 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43309
I20260811 06:35:24.376700 30849 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43309 every 8 connection(s)
I20260811 06:35:24.378350 30740 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
I20260811 06:35:24.380654 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 30740
I20260811 06:35:24.380888 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786429755927508-29631-0/minicluster-data/master-0/wal/instance
I20260811 06:35:24.381438 29631 external_mini_cluster.cc:1428] Setting key b75c03a785cab405e18a664e7a469ab8
I20260811 06:35:24.383121 30852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 06:35:24 dist-test-slave-gkw7 krb5kdc[30707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430123, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 06:35:24.398286 30852 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e86f93f0ba0e45a693e7d377f403500c: Bootstrap starting.
I20260811 06:35:24.402232 30852 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e86f93f0ba0e45a693e7d377f403500c: Neither blocks nor log segments found. Creating new log.
I20260811 06:35:24.403512 30852 log.cc:824] T 00000000000000000000000000000000 P e86f93f0ba0e45a693e7d377f403500c: Log is configured to *not* fsync() on all Append() calls
I20260811 06:35:24.407650 30852 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e86f93f0ba0e45a693e7d377f403500c: No bootstrap required, opened a new log
I20260811 06:35:24.408830 30855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:24.383557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.125:58421 (local address 127.0.0.1:43309)
0811 06:35:24.384381 (+ 824us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:24.384398 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:24.384453 (+ 55us) server_negotiation.cc:407] Connection header received
0811 06:35:24.386708 (+ 2255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:24.386738 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:24.387359 (+ 621us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:24.387833 (+ 474us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:24.389334 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:24.391464 (+ 2130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:24.392769 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:24.394026 (+ 1257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:24.394253 (+ 227us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:24.399354 (+ 5101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:35:24.399363 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:35:24.399380 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:35:24.399423 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:35:24.403200 (+ 3777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:35:24.404547 (+ 1347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:35:24.404559 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:35:24.404569 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:35:24.404811 (+ 242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:35:24.405490 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:35:24.405495 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:35:24.405498 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:35:24.406229 (+ 731us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:35:24.406499 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0811 06:35:24.407228 (+ 729us) server_negotiation.cc:299] Negotiation successful
0811 06:35:24.407651 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":580,"thread_start_us":338,"threads_started":1}
I20260811 06:35:24.413699 29631 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 06:35:24.417735 30852 raft_consensus.cc:348] T 00000000000000000000000000000000 P e86f93f0ba0e45a693e7d377f403500c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e86f93f0ba0e45a693e7d377f403500c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43309 } }
I20260811 06:35:24.418187 30852 raft_consensus.cc:374] T 00000000000000000000000000000000 P e86f93f0ba0e45a693e7d377f403500c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:35:24.418309 30852 raft_consensus.cc:729] T 00000000000000000000000000000000 P e86f93f0ba0e45a693e7d377f403500c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e86f93f0ba0e45a693e7d377f403500c, State: Initialized, Role: FOLLOWER
I20260811 06:35:24.419047 30852 consensus_queue.cc:260] T 00000000000000000000000000000000 P e86f93f0ba0e45a693e7d377f403500c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e86f93f0ba0e45a693e7d377f403500c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43309 } }
I20260811 06:35:24.419356 30852 raft_consensus.cc:388] T 00000000000000000000000000000000 P e86f93f0ba0e45a693e7d377f403500c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:35:24.419540 30852 raft_consensus.cc:482] T 00000000000000000000000000000000 P e86f93f0ba0e45a693e7d377f403500c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:35:24.419759 30852 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e86f93f0ba0e45a693e7d377f403500c [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:35:24.423601 30852 raft_consensus.cc:504] T 00000000000000000000000000000000 P e86f93f0ba0e45a693e7d377f403500c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e86f93f0ba0e45a693e7d377f403500c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43309 } }
I20260811 06:35:24.424364 30852 leader_election.cc:302] T 00000000000000000000000000000000 P e86f93f0ba0e45a693e7d377f403500c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e86f93f0ba0e45a693e7d377f403500c; no voters:
I20260811 06:35:24.425203 30852 leader_election.cc:288] T 00000000000000000000000000000000 P e86f93f0ba0e45a693e7d377f403500c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:35:24.425325 30870 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e86f93f0ba0e45a693e7d377f403500c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:35:24.426692 30870 raft_consensus.cc:686] T 00000000000000000000000000000000 P e86f93f0ba0e45a693e7d377f403500c [term 1 LEADER]: Becoming Leader. State: Replica: e86f93f0ba0e45a693e7d377f403500c, State: Running, Role: LEADER
I20260811 06:35:24.427457 30870 consensus_queue.cc:237] T 00000000000000000000000000000000 P e86f93f0ba0e45a693e7d377f403500c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e86f93f0ba0e45a693e7d377f403500c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43309 } }
I20260811 06:35:24.428156 30852 sys_catalog.cc:564] T 00000000000000000000000000000000 P e86f93f0ba0e45a693e7d377f403500c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:35:24.431377 30872 sys_catalog.cc:455] T 00000000000000000000000000000000 P e86f93f0ba0e45a693e7d377f403500c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e86f93f0ba0e45a693e7d377f403500c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e86f93f0ba0e45a693e7d377f403500c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43309 } } }
I20260811 06:35:24.431828 30872 sys_catalog.cc:458] T 00000000000000000000000000000000 P e86f93f0ba0e45a693e7d377f403500c [sys.catalog]: This master's current role is: LEADER
I20260811 06:35:24.432545 30874 sys_catalog.cc:455] T 00000000000000000000000000000000 P e86f93f0ba0e45a693e7d377f403500c [sys.catalog]: SysCatalogTable state changed. Reason: New leader e86f93f0ba0e45a693e7d377f403500c. Latest consensus state: current_term: 1 leader_uuid: "e86f93f0ba0e45a693e7d377f403500c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e86f93f0ba0e45a693e7d377f403500c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43309 } } }
I20260811 06:35:24.433454 30874 sys_catalog.cc:458] T 00000000000000000000000000000000 P e86f93f0ba0e45a693e7d377f403500c [sys.catalog]: This master's current role is: LEADER
I20260811 06:35:24.437599 30877 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:35:24.441460 30855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:24.418985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.125:50407 (local address 127.0.0.1:43309)
0811 06:35:24.419219 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:24.419230 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:24.419969 (+ 739us) server_negotiation.cc:407] Connection header received
0811 06:35:24.420157 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:24.420161 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:24.420240 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:24.420336 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:24.422755 (+ 2419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:24.423840 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:24.425673 (+ 1833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:24.426852 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:24.427037 (+ 185us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:24.430324 (+ 3287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:35:24.430335 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:35:24.430344 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:35:24.430405 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:35:24.435736 (+ 5331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:35:24.437193 (+ 1457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:35:24.437203 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:35:24.437210 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:35:24.437434 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:35:24.438325 (+ 891us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:35:24.438339 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:35:24.438346 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:35:24.438927 (+ 581us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:35:24.439276 (+ 349us) server_negotiation.cc:1035] Waiting for connection context
0811 06:35:24.440521 (+ 1245us) server_negotiation.cc:299] Negotiation successful
0811 06:35:24.440808 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 06:35:24.447119 30877 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:35:24.458205 30877 catalog_manager.cc:1347] Generated new cluster ID: 09a98e99e6a54090a6c687f418ca3e92
I20260811 06:35:24.458356 30877 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:35:24.469003 30877 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:35:24.470644 30877 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:35:24.487921 30877 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e86f93f0ba0e45a693e7d377f403500c: Generated new TSK 0
I20260811 06:35:24.489117 30877 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:35:24.496145 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 30740
I20260811 06:35:24.498265 30838 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:35:24.590945 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 30740
W20260811 06:35:24.614970 30704 connection.cc:535] client connection to 127.0.0.1:43309 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43309): Connection reset by peer (error 104)
2026-08-11T06:35:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2415 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786429755927508-29631-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:35:24 dist-test-slave-gkw7 krb5kdc[30907](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:35:24 dist-test-slave-gkw7 krb5kdc[30907](info): set up 2 sockets
Aug 11 06:35:24 dist-test-slave-gkw7 krb5kdc[30907](info): commencing operation
krb5kdc: starting...
W20260811 06:35:26.033967 29631 mini_kdc.cc:121] Time spent starting KDC: real 1.400s user 0.000s sys 0.050s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786429755927508-29631-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786429755927508-29631-0/krb5kdc/test-user.keytab.
Aug 11 06:35:26 dist-test-slave-gkw7 krb5kdc[30907](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430126, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:35:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:35:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786429755927508-29631-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786429755927508-29631-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 06:35:26.388516 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786429755927508-29631-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786429755927508-29631-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786429755927508-29631-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37539
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786429755927508-29631-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37539
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:35:26.747750 30925 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:35:26.748049 30925 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:35:26.748142 30925 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:35:26.755330 30925 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:35:26.755450 30925 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:35:26.755497 30925 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:35:26.755561 30925 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:35:26.755606 30925 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:35:26.765786 30925 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786429755927508-29631-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786429755927508-29631-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37539
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786429755927508-29631-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37539
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786429755927508-29631-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:35:26.767115 30925 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:35:26.768433 30925 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:35:26.780789 30932 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:35:26.780827 30938 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:35:26.780859 30936 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:35:26.781601 30925 server_base.cc:1034] running on GCE node
I20260811 06:35:26.782413 30925 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:35:26.783819 30925 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:35:26.785029 30925 hybrid_clock.cc:648] HybridClock initialized: now 1786430126784993 us; error 27 us; skew 500 ppm
I20260811 06:35:26.785393 30925 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:35:26 dist-test-slave-gkw7 krb5kdc[30907](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430126, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:35:26.788692 30925 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 06:35:26.790333 30925 webserver.cc:460] Webserver started at http://127.0.0.1:43015/ using document root <none> and password file <none>
I20260811 06:35:26.791042 30925 fs_manager.cc:359] Metadata directory not provided
I20260811 06:35:26.791199 30925 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:35:26.791532 30925 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:35:26.794693 30925 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786429755927508-29631-0/minicluster-data/master-0/data/instance:
uuid: "c679d528d95a47c2a36cdc95a6194583"
format_stamp: "Formatted at 2026-08-11 06:35:26 on dist-test-slave-gkw7"
server_key: "2ff6a86330d32f6a7543eb06787e0853"
server_key_iv: "0e368a9fd9cdcaefb76588fd4b12ed55"
server_key_version: "encryptionkey@0"
I20260811 06:35:26.795471 30925 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786429755927508-29631-0/minicluster-data/master-0/wal/instance:
uuid: "c679d528d95a47c2a36cdc95a6194583"
format_stamp: "Formatted at 2026-08-11 06:35:26 on dist-test-slave-gkw7"
server_key: "2ff6a86330d32f6a7543eb06787e0853"
server_key_iv: "0e368a9fd9cdcaefb76588fd4b12ed55"
server_key_version: "encryptionkey@0"
I20260811 06:35:26.800756 30925 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260811 06:35:26.804396 30947 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:35:26.805869 30925 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:35:26.806066 30925 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786429755927508-29631-0/minicluster-data/master-0/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786429755927508-29631-0/minicluster-data/master-0/wal
uuid: "c679d528d95a47c2a36cdc95a6194583"
format_stamp: "Formatted at 2026-08-11 06:35:26 on dist-test-slave-gkw7"
server_key: "2ff6a86330d32f6a7543eb06787e0853"
server_key_iv: "0e368a9fd9cdcaefb76588fd4b12ed55"
server_key_version: "encryptionkey@0"
I20260811 06:35:26.806288 30925 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786429755927508-29631-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786429755927508-29631-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786429755927508-29631-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:35:26.834008 30925 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:35:26.838558 30925 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:35:26.838879 30925 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:35:26.858580 30925 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37539
I20260811 06:35:26.858604 31036 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37539 every 8 connection(s)
I20260811 06:35:26.860078 30925 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
I20260811 06:35:26.863500 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 30925
I20260811 06:35:26.863695 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786429755927508-29631-0/minicluster-data/master-0/wal/instance
I20260811 06:35:26.864285 29631 external_mini_cluster.cc:1428] Setting key 05dc82491af905405f69c12c52542279
I20260811 06:35:26.864394 31037 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:35:26.878854 31037 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c679d528d95a47c2a36cdc95a6194583: Bootstrap starting.
I20260811 06:35:26.879335 31043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:26.866306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.125:34049 (local address 127.0.0.1:37539)
0811 06:35:26.867246 (+ 940us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:26.867258 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:26.867304 (+ 46us) server_negotiation.cc:407] Connection header received
0811 06:35:26.869401 (+ 2097us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:26.869426 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:26.870066 (+ 640us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:26.870445 (+ 379us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:26.871059 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:26.872566 (+ 1507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:26.874542 (+ 1976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:26.875302 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:26.875449 (+ 147us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:26.876826 (+ 1377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:35:26.876833 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:35:26.876847 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:35:26.877395 (+ 548us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:35:26.877476 (+ 81us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:35:26.877500 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:35:26.877706 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0811 06:35:26.878027 (+ 321us) server_negotiation.cc:299] Negotiation successful
0811 06:35:26.878248 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":701,"thread_start_us":428,"threads_started":1}
I20260811 06:35:26.882272 31037 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c679d528d95a47c2a36cdc95a6194583: Neither blocks nor log segments found. Creating new log.
I20260811 06:35:26.883255 29631 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 06:35:26.883448 31037 log.cc:824] T 00000000000000000000000000000000 P c679d528d95a47c2a36cdc95a6194583: Log is configured to *not* fsync() on all Append() calls
I20260811 06:35:26.886780 31037 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c679d528d95a47c2a36cdc95a6194583: No bootstrap required, opened a new log
I20260811 06:35:26.895691 31043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:26.888674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.125:47887 (local address 127.0.0.1:37539)
0811 06:35:26.888834 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:26.888839 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:26.889240 (+ 401us) server_negotiation.cc:407] Connection header received
0811 06:35:26.889434 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:26.889438 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:26.889504 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:26.889648 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:26.890256 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:26.891025 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:26.892970 (+ 1945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:26.893431 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:26.893583 (+ 152us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:26.894859 (+ 1276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:35:26.894864 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:35:26.894867 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:35:26.894885 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:35:26.894926 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:35:26.894943 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:35:26.895032 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 06:35:26.895260 (+ 228us) server_negotiation.cc:299] Negotiation successful
0811 06:35:26.895309 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 06:35:26.897282 31037 raft_consensus.cc:348] T 00000000000000000000000000000000 P c679d528d95a47c2a36cdc95a6194583 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c679d528d95a47c2a36cdc95a6194583" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37539 } }
I20260811 06:35:26.897773 31037 raft_consensus.cc:374] T 00000000000000000000000000000000 P c679d528d95a47c2a36cdc95a6194583 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:35:26.897900 31037 raft_consensus.cc:729] T 00000000000000000000000000000000 P c679d528d95a47c2a36cdc95a6194583 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c679d528d95a47c2a36cdc95a6194583, State: Initialized, Role: FOLLOWER
I20260811 06:35:26.898514 31037 consensus_queue.cc:260] T 00000000000000000000000000000000 P c679d528d95a47c2a36cdc95a6194583 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c679d528d95a47c2a36cdc95a6194583" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37539 } }
I20260811 06:35:26.898761 31037 raft_consensus.cc:388] T 00000000000000000000000000000000 P c679d528d95a47c2a36cdc95a6194583 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:35:26.898938 31037 raft_consensus.cc:482] T 00000000000000000000000000000000 P c679d528d95a47c2a36cdc95a6194583 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:35:26.899134 31037 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c679d528d95a47c2a36cdc95a6194583 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:35:26.901487 31037 raft_consensus.cc:504] T 00000000000000000000000000000000 P c679d528d95a47c2a36cdc95a6194583 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c679d528d95a47c2a36cdc95a6194583" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37539 } }
I20260811 06:35:26.902102 31037 leader_election.cc:302] T 00000000000000000000000000000000 P c679d528d95a47c2a36cdc95a6194583 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c679d528d95a47c2a36cdc95a6194583; no voters:
I20260811 06:35:26.902716 31037 leader_election.cc:288] T 00000000000000000000000000000000 P c679d528d95a47c2a36cdc95a6194583 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:35:26.902884 31057 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c679d528d95a47c2a36cdc95a6194583 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:35:26.903617 31057 raft_consensus.cc:686] T 00000000000000000000000000000000 P c679d528d95a47c2a36cdc95a6194583 [term 1 LEADER]: Becoming Leader. State: Replica: c679d528d95a47c2a36cdc95a6194583, State: Running, Role: LEADER
I20260811 06:35:26.904242 31057 consensus_queue.cc:237] T 00000000000000000000000000000000 P c679d528d95a47c2a36cdc95a6194583 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c679d528d95a47c2a36cdc95a6194583" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37539 } }
I20260811 06:35:26.905030 31037 sys_catalog.cc:564] T 00000000000000000000000000000000 P c679d528d95a47c2a36cdc95a6194583 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:35:26.907649 31058 sys_catalog.cc:455] T 00000000000000000000000000000000 P c679d528d95a47c2a36cdc95a6194583 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c679d528d95a47c2a36cdc95a6194583" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c679d528d95a47c2a36cdc95a6194583" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37539 } } }
I20260811 06:35:26.907740 31060 sys_catalog.cc:455] T 00000000000000000000000000000000 P c679d528d95a47c2a36cdc95a6194583 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c679d528d95a47c2a36cdc95a6194583. Latest consensus state: current_term: 1 leader_uuid: "c679d528d95a47c2a36cdc95a6194583" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c679d528d95a47c2a36cdc95a6194583" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37539 } } }
I20260811 06:35:26.908049 31058 sys_catalog.cc:458] T 00000000000000000000000000000000 P c679d528d95a47c2a36cdc95a6194583 [sys.catalog]: This master's current role is: LEADER
I20260811 06:35:26.908165 31060 sys_catalog.cc:458] T 00000000000000000000000000000000 P c679d528d95a47c2a36cdc95a6194583 [sys.catalog]: This master's current role is: LEADER
I20260811 06:35:26.909153 31064 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:35:26.916071 31064 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:35:26.926292 31064 catalog_manager.cc:1347] Generated new cluster ID: bcdb9f66c5f241ea833555964e09d6b3
I20260811 06:35:26.926430 31064 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:35:26.941843 31064 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:35:26.943209 31064 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:35:26.959720 31064 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c679d528d95a47c2a36cdc95a6194583: Generated new TSK 0
I20260811 06:35:26.960750 31064 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:35:26.980240 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 30925
W20260811 06:35:26.981756 31030 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.125:34049
W20260811 06:35:26.982668 29631 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 30925: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 06:35:26.982841 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 30925
W20260811 06:35:27.007674 30905 connection.cc:535] client connection to 127.0.0.1:37539 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37539): Connection reset by peer (error 104)
2026-08-11T06:35:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2396 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786429755927508-29631-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:35:27 dist-test-slave-gkw7 krb5kdc[31089](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:35:27 dist-test-slave-gkw7 krb5kdc[31089](info): set up 2 sockets
Aug 11 06:35:27 dist-test-slave-gkw7 krb5kdc[31089](info): commencing operation
krb5kdc: starting...
W20260811 06:35:28.365983 29631 mini_kdc.cc:121] Time spent starting KDC: real 1.335s user 0.004s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786429755927508-29631-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786429755927508-29631-0/krb5kdc/test-user.keytab.
Aug 11 06:35:28 dist-test-slave-gkw7 krb5kdc[31089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430128, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:35:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:35:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786429755927508-29631-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786429755927508-29631-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 06:35:28.714152 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786429755927508-29631-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786429755927508-29631-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786429755927508-29631-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44595
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786429755927508-29631-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44595
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:35:29.070233 31113 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:35:29.070590 31113 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:35:29.070770 31113 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:35:29.078162 31113 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:35:29.078366 31113 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:35:29.078473 31113 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:35:29.078593 31113 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:35:29.078688 31113 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:35:29.088881 31113 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786429755927508-29631-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786429755927508-29631-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44595
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786429755927508-29631-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44595
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786429755927508-29631-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:35:29.090351 31113 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:35:29.091751 31113 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:35:29.104377 31124 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:35:29.104439 31128 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:35:29.104561 31125 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:35:29.105747 31113 server_base.cc:1034] running on GCE node
I20260811 06:35:29.106627 31113 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:35:29.108110 31113 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:35:29.109398 31113 hybrid_clock.cc:648] HybridClock initialized: now 1786430129109365 us; error 47 us; skew 500 ppm
I20260811 06:35:29.109835 31113 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:35:29 dist-test-slave-gkw7 krb5kdc[31089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430129, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:35:29.118170 31113 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 06:35:29.119946 31113 webserver.cc:460] Webserver started at http://127.0.0.1:34617/ using document root <none> and password file <none>
I20260811 06:35:29.120679 31113 fs_manager.cc:359] Metadata directory not provided
I20260811 06:35:29.120795 31113 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:35:29.121068 31113 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:35:29.124421 31113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786429755927508-29631-0/minicluster-data/master-0/data/instance:
uuid: "1d24c3f89cd34da7b02c8c6e5f4fe708"
format_stamp: "Formatted at 2026-08-11 06:35:29 on dist-test-slave-gkw7"
server_key: "3d42d0937146f68eb7d7777b92bd8d3a"
server_key_iv: "5c277c74c0f5d36cf172e232c1892a0d"
server_key_version: "encryptionkey@0"
I20260811 06:35:29.125216 31113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786429755927508-29631-0/minicluster-data/master-0/wal/instance:
uuid: "1d24c3f89cd34da7b02c8c6e5f4fe708"
format_stamp: "Formatted at 2026-08-11 06:35:29 on dist-test-slave-gkw7"
server_key: "3d42d0937146f68eb7d7777b92bd8d3a"
server_key_iv: "5c277c74c0f5d36cf172e232c1892a0d"
server_key_version: "encryptionkey@0"
I20260811 06:35:29.130980 31113 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 06:35:29.134718 31140 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:35:29.136340 31113 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 06:35:29.136561 31113 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786429755927508-29631-0/minicluster-data/master-0/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786429755927508-29631-0/minicluster-data/master-0/wal
uuid: "1d24c3f89cd34da7b02c8c6e5f4fe708"
format_stamp: "Formatted at 2026-08-11 06:35:29 on dist-test-slave-gkw7"
server_key: "3d42d0937146f68eb7d7777b92bd8d3a"
server_key_iv: "5c277c74c0f5d36cf172e232c1892a0d"
server_key_version: "encryptionkey@0"
I20260811 06:35:29.136788 31113 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786429755927508-29631-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786429755927508-29631-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786429755927508-29631-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:35:29.143039 31113 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:35:29.147423 31113 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:35:29.147737 31113 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:35:29.167220 31113 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44595
I20260811 06:35:29.167316 31237 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44595 every 8 connection(s)
I20260811 06:35:29.168797 31113 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
I20260811 06:35:29.169950 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 31113
I20260811 06:35:29.170149 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786429755927508-29631-0/minicluster-data/master-0/wal/instance
I20260811 06:35:29.170689 29631 external_mini_cluster.cc:1428] Setting key 1768fab95b6cdca49dfd5d51b897a710
I20260811 06:35:29.173741 31239 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:35:29.179482 31242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:29.172654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.125:50851 (local address 127.0.0.1:44595)
0811 06:35:29.173499 (+ 845us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:29.173512 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:29.173627 (+ 115us) server_negotiation.cc:407] Connection header received
0811 06:35:29.175853 (+ 2226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:29.175882 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:29.176252 (+ 370us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:29.176617 (+ 365us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:29.177747 (+ 1130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:35:29.177753 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:35:29.177766 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:35:29.177781 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:35:29.177864 (+ 83us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:35:29.177884 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:35:29.178119 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0811 06:35:29.178225 (+ 106us) server_negotiation.cc:299] Negotiation successful
0811 06:35:29.178480 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":607,"thread_start_us":392,"threads_started":1}
I20260811 06:35:29.183553 29631 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 06:35:29.187212 31239 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d24c3f89cd34da7b02c8c6e5f4fe708: Bootstrap starting.
I20260811 06:35:29.191555 31239 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1d24c3f89cd34da7b02c8c6e5f4fe708: Neither blocks nor log segments found. Creating new log.
I20260811 06:35:29.191840 31242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:29.189192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.125:42249 (local address 127.0.0.1:44595)
0811 06:35:29.189350 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:29.189354 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:29.189742 (+ 388us) server_negotiation.cc:407] Connection header received
0811 06:35:29.189905 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:29.189908 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:29.189956 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:29.190045 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:29.191011 (+ 966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:35:29.191015 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:35:29.191018 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:35:29.191032 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:35:29.191098 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:35:29.191114 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:35:29.191203 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 06:35:29.191414 (+ 211us) server_negotiation.cc:299] Negotiation successful
0811 06:35:29.191473 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 06:35:29.192832 31239 log.cc:824] T 00000000000000000000000000000000 P 1d24c3f89cd34da7b02c8c6e5f4fe708: Log is configured to *not* fsync() on all Append() calls
I20260811 06:35:29.195602 31239 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d24c3f89cd34da7b02c8c6e5f4fe708: No bootstrap required, opened a new log
I20260811 06:35:29.202976 31239 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1d24c3f89cd34da7b02c8c6e5f4fe708 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d24c3f89cd34da7b02c8c6e5f4fe708" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44595 } }
I20260811 06:35:29.203361 31239 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1d24c3f89cd34da7b02c8c6e5f4fe708 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:35:29.203474 31239 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1d24c3f89cd34da7b02c8c6e5f4fe708 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d24c3f89cd34da7b02c8c6e5f4fe708, State: Initialized, Role: FOLLOWER
I20260811 06:35:29.204049 31239 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1d24c3f89cd34da7b02c8c6e5f4fe708 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d24c3f89cd34da7b02c8c6e5f4fe708" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44595 } }
I20260811 06:35:29.204306 31239 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1d24c3f89cd34da7b02c8c6e5f4fe708 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:35:29.204459 31239 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1d24c3f89cd34da7b02c8c6e5f4fe708 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:35:29.204646 31239 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1d24c3f89cd34da7b02c8c6e5f4fe708 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:35:29.206920 31239 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1d24c3f89cd34da7b02c8c6e5f4fe708 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d24c3f89cd34da7b02c8c6e5f4fe708" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44595 } }
I20260811 06:35:29.207432 31239 leader_election.cc:302] T 00000000000000000000000000000000 P 1d24c3f89cd34da7b02c8c6e5f4fe708 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1d24c3f89cd34da7b02c8c6e5f4fe708; no voters:
I20260811 06:35:29.208052 31239 leader_election.cc:288] T 00000000000000000000000000000000 P 1d24c3f89cd34da7b02c8c6e5f4fe708 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:35:29.208271 31253 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1d24c3f89cd34da7b02c8c6e5f4fe708 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:35:29.209048 31253 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1d24c3f89cd34da7b02c8c6e5f4fe708 [term 1 LEADER]: Becoming Leader. State: Replica: 1d24c3f89cd34da7b02c8c6e5f4fe708, State: Running, Role: LEADER
I20260811 06:35:29.209731 31253 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1d24c3f89cd34da7b02c8c6e5f4fe708 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d24c3f89cd34da7b02c8c6e5f4fe708" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44595 } }
I20260811 06:35:29.210309 31239 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1d24c3f89cd34da7b02c8c6e5f4fe708 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:35:29.213251 31255 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d24c3f89cd34da7b02c8c6e5f4fe708 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1d24c3f89cd34da7b02c8c6e5f4fe708. Latest consensus state: current_term: 1 leader_uuid: "1d24c3f89cd34da7b02c8c6e5f4fe708" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d24c3f89cd34da7b02c8c6e5f4fe708" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44595 } } }
I20260811 06:35:29.213833 31255 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d24c3f89cd34da7b02c8c6e5f4fe708 [sys.catalog]: This master's current role is: LEADER
I20260811 06:35:29.213647 31254 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d24c3f89cd34da7b02c8c6e5f4fe708 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1d24c3f89cd34da7b02c8c6e5f4fe708" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d24c3f89cd34da7b02c8c6e5f4fe708" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44595 } } }
I20260811 06:35:29.213948 31254 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d24c3f89cd34da7b02c8c6e5f4fe708 [sys.catalog]: This master's current role is: LEADER
I20260811 06:35:29.216010 31266 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:35:29.221321 31266 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:35:29.232256 31266 catalog_manager.cc:1347] Generated new cluster ID: e7ff4f9f970a4d4589411275d0f6a062
I20260811 06:35:29.232364 31266 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:35:29.244331 31266 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:35:29.244792 31266 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:35:29.256245 31266 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1d24c3f89cd34da7b02c8c6e5f4fe708: Generated new TSK 0
I20260811 06:35:29.257369 31266 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:35:29.268594 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 31113
W20260811 06:35:29.270141 31231 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.125:50851
W20260811 06:35:29.270962 29631 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 31113: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 06:35:29.271158 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 31113
2026-08-11T06:35:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (2292 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786429755927508-29631-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:35:29 dist-test-slave-gkw7 krb5kdc[31282](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:35:29 dist-test-slave-gkw7 krb5kdc[31282](info): set up 2 sockets
Aug 11 06:35:29 dist-test-slave-gkw7 krb5kdc[31282](info): commencing operation
krb5kdc: starting...
W20260811 06:35:30.696410 29631 mini_kdc.cc:121] Time spent starting KDC: real 1.373s user 0.000s sys 0.052s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786429755927508-29631-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786429755927508-29631-0/krb5kdc/test-user.keytab.
Aug 11 06:35:30 dist-test-slave-gkw7 krb5kdc[31282](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430130, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:35:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:35:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.239.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.239.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.239.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab.
Entry for principal kudu/127.28.239.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab.
I20260811 06:35:31.053941 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786429755927508-29631-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786429755927508-29631-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786429755927508-29631-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:38193
--webserver_interface=127.28.239.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab
--principal=kudu/127.28.239.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:41219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.239.254:38193
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:35:31.424651 31299 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:35:31.424978 31299 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:35:31.425148 31299 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:35:31.432824 31299 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:35:31.432955 31299 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:35:31.433075 31299 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:35:31.433195 31299 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:35:31.433283 31299 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:35:31.443607 31299 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:41219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786429755927508-29631-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786429755927508-29631-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.239.254:38193
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab
--principal=kudu/127.28.239.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:38193
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.239.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786429755927508-29631-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:35:31.445024 31299 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:35:31.446538 31299 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:35:31.459465 31309 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:35:31.459463 31312 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:35:31.459496 31308 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:35:31.460358 31299 server_base.cc:1034] running on GCE node
I20260811 06:35:31.461236 31299 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:35:31.462834 31299 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:35:31.464118 31299 hybrid_clock.cc:648] HybridClock initialized: now 1786430131464047 us; error 55 us; skew 500 ppm
I20260811 06:35:31.464570 31299 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:35:31 dist-test-slave-gkw7 krb5kdc[31282](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430131, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:35:31.468209 31299 init.cc:377] Logged in from keytab as kudu/127.28.239.254@KRBTEST.COM (short username kudu)
I20260811 06:35:31.470093 31299 webserver.cc:460] Webserver started at http://127.28.239.254:40561/ using document root <none> and password file <none>
I20260811 06:35:31.470853 31299 fs_manager.cc:359] Metadata directory not provided
I20260811 06:35:31.470974 31299 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:35:31.471267 31299 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:35:31.474642 31299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786429755927508-29631-0/minicluster-data/master-0/data/instance:
uuid: "c5f240f6164c49b9a6a4b6dbbbbe8613"
format_stamp: "Formatted at 2026-08-11 06:35:31 on dist-test-slave-gkw7"
server_key: "5ec20713b0c14c4610a7e288c412a085"
server_key_iv: "03826c6828ccc6405fd6dd7f507ab8c6"
server_key_version: "encryptionkey@0"
I20260811 06:35:31.475437 31299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786429755927508-29631-0/minicluster-data/master-0/wal/instance:
uuid: "c5f240f6164c49b9a6a4b6dbbbbe8613"
format_stamp: "Formatted at 2026-08-11 06:35:31 on dist-test-slave-gkw7"
server_key: "5ec20713b0c14c4610a7e288c412a085"
server_key_iv: "03826c6828ccc6405fd6dd7f507ab8c6"
server_key_version: "encryptionkey@0"
I20260811 06:35:31.481381 31299 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 06:35:31.485159 31323 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:35:31.486671 31299 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:35:31.486881 31299 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786429755927508-29631-0/minicluster-data/master-0/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786429755927508-29631-0/minicluster-data/master-0/wal
uuid: "c5f240f6164c49b9a6a4b6dbbbbe8613"
format_stamp: "Formatted at 2026-08-11 06:35:31 on dist-test-slave-gkw7"
server_key: "5ec20713b0c14c4610a7e288c412a085"
server_key_iv: "03826c6828ccc6405fd6dd7f507ab8c6"
server_key_version: "encryptionkey@0"
I20260811 06:35:31.487097 31299 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786429755927508-29631-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786429755927508-29631-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786429755927508-29631-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:35:31.506232 31299 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:35:31.510807 31299 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:35:31.511130 31299 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:35:31.531209 31299 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.254:38193
I20260811 06:35:31.531250 31402 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.254:38193 every 8 connection(s)
I20260811 06:35:31.532716 31299 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
I20260811 06:35:31.536697 31405 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:35:31.541085 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 31299
I20260811 06:35:31.541283 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786429755927508-29631-0/minicluster-data/master-0/wal/instance
I20260811 06:35:31.541941 29631 external_mini_cluster.cc:1428] Setting key 74e82d399aeb666c3a8dc8a2ee388aaf
I20260811 06:35:31.548918 31405 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5f240f6164c49b9a6a4b6dbbbbe8613: Bootstrap starting.
I20260811 06:35:31.552753 31405 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c5f240f6164c49b9a6a4b6dbbbbe8613: Neither blocks nor log segments found. Creating new log.
I20260811 06:35:31.554039 31405 log.cc:824] T 00000000000000000000000000000000 P c5f240f6164c49b9a6a4b6dbbbbe8613: Log is configured to *not* fsync() on all Append() calls
I20260811 06:35:31.557093 31405 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5f240f6164c49b9a6a4b6dbbbbe8613: No bootstrap required, opened a new log
Aug 11 06:35:31 dist-test-slave-gkw7 krb5kdc[31282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430130, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:35:31.565001 31405 raft_consensus.cc:348] T 00000000000000000000000000000000 P c5f240f6164c49b9a6a4b6dbbbbe8613 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5f240f6164c49b9a6a4b6dbbbbe8613" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 38193 } }
I20260811 06:35:31.565474 31405 raft_consensus.cc:374] T 00000000000000000000000000000000 P c5f240f6164c49b9a6a4b6dbbbbe8613 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:35:31.565647 31405 raft_consensus.cc:729] T 00000000000000000000000000000000 P c5f240f6164c49b9a6a4b6dbbbbe8613 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5f240f6164c49b9a6a4b6dbbbbe8613, State: Initialized, Role: FOLLOWER
I20260811 06:35:31.566270 31405 consensus_queue.cc:260] T 00000000000000000000000000000000 P c5f240f6164c49b9a6a4b6dbbbbe8613 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5f240f6164c49b9a6a4b6dbbbbe8613" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 38193 } }
I20260811 06:35:31.566545 31405 raft_consensus.cc:388] T 00000000000000000000000000000000 P c5f240f6164c49b9a6a4b6dbbbbe8613 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:35:31.566715 31405 raft_consensus.cc:482] T 00000000000000000000000000000000 P c5f240f6164c49b9a6a4b6dbbbbe8613 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:35:31.566912 31405 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c5f240f6164c49b9a6a4b6dbbbbe8613 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:35:31.569331 31405 raft_consensus.cc:504] T 00000000000000000000000000000000 P c5f240f6164c49b9a6a4b6dbbbbe8613 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5f240f6164c49b9a6a4b6dbbbbe8613" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 38193 } }
I20260811 06:35:31.569952 31405 leader_election.cc:302] T 00000000000000000000000000000000 P c5f240f6164c49b9a6a4b6dbbbbe8613 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c5f240f6164c49b9a6a4b6dbbbbe8613; no voters:
I20260811 06:35:31.570737 31405 leader_election.cc:288] T 00000000000000000000000000000000 P c5f240f6164c49b9a6a4b6dbbbbe8613 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:35:31.570874 31413 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c5f240f6164c49b9a6a4b6dbbbbe8613 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:35:31.572531 31409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:31.543898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44295 (local address 127.28.239.254:38193)
0811 06:35:31.544796 (+ 898us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:31.544810 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:31.544858 (+ 48us) server_negotiation.cc:407] Connection header received
0811 06:35:31.547307 (+ 2449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:31.547333 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:31.547777 (+ 444us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:31.548118 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:31.550067 (+ 1949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:31.551444 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:31.553347 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:31.554112 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:31.554317 (+ 205us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:31.563213 (+ 8896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:35:31.563222 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:35:31.563238 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:35:31.563283 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:35:31.566987 (+ 3704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:35:31.567915 (+ 928us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:35:31.567927 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:35:31.567937 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:35:31.568123 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:35:31.568671 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:35:31.568676 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:35:31.568680 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:35:31.569268 (+ 588us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:35:31.569655 (+ 387us) server_negotiation.cc:1035] Waiting for connection context
0811 06:35:31.570192 (+ 537us) server_negotiation.cc:299] Negotiation successful
0811 06:35:31.570584 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":668,"thread_start_us":451,"threads_started":1}
I20260811 06:35:31.573113 31405 sys_catalog.cc:564] T 00000000000000000000000000000000 P c5f240f6164c49b9a6a4b6dbbbbe8613 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:35:31.574034 31413 raft_consensus.cc:686] T 00000000000000000000000000000000 P c5f240f6164c49b9a6a4b6dbbbbe8613 [term 1 LEADER]: Becoming Leader. State: Replica: c5f240f6164c49b9a6a4b6dbbbbe8613, State: Running, Role: LEADER
I20260811 06:35:31.574784 31413 consensus_queue.cc:237] T 00000000000000000000000000000000 P c5f240f6164c49b9a6a4b6dbbbbe8613 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5f240f6164c49b9a6a4b6dbbbbe8613" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 38193 } }
I20260811 06:35:31.578572 29631 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 06:35:31.583038 31414 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5f240f6164c49b9a6a4b6dbbbbe8613 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c5f240f6164c49b9a6a4b6dbbbbe8613" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5f240f6164c49b9a6a4b6dbbbbe8613" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 38193 } } }
I20260811 06:35:31.583527 31414 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5f240f6164c49b9a6a4b6dbbbbe8613 [sys.catalog]: This master's current role is: LEADER
I20260811 06:35:31.585846 31415 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5f240f6164c49b9a6a4b6dbbbbe8613 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c5f240f6164c49b9a6a4b6dbbbbe8613. Latest consensus state: current_term: 1 leader_uuid: "c5f240f6164c49b9a6a4b6dbbbbe8613" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5f240f6164c49b9a6a4b6dbbbbe8613" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 38193 } } }
I20260811 06:35:31.586930 31415 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5f240f6164c49b9a6a4b6dbbbbe8613 [sys.catalog]: This master's current role is: LEADER
I20260811 06:35:31.589243 31431 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:35:31.602423 31431 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:35:31.619534 31431 catalog_manager.cc:1347] Generated new cluster ID: ed2bb69985d443b59a4aa0bb2b4f909d
I20260811 06:35:31.619658 31431 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:35:31.626384 31409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:31.600087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36023 (local address 127.28.239.254:38193)
0811 06:35:31.600298 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:31.600311 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:31.606805 (+ 6494us) server_negotiation.cc:407] Connection header received
0811 06:35:31.607370 (+ 565us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:31.607375 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:31.607465 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:31.607561 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:31.609845 (+ 2284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:31.610971 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:31.613362 (+ 2391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:31.614370 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:31.614504 (+ 134us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:31.618842 (+ 4338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:35:31.618850 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:35:31.618855 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:35:31.618895 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:35:31.623004 (+ 4109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:35:31.623975 (+ 971us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:35:31.623981 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:35:31.623984 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:35:31.624132 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:35:31.624630 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:35:31.624635 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:35:31.624639 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:35:31.625080 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:35:31.625354 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0811 06:35:31.625781 (+ 427us) server_negotiation.cc:299] Negotiation successful
0811 06:35:31.625996 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 06:35:31.636144 31431 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:35:31.638095 31431 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:35:31.654433 31431 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c5f240f6164c49b9a6a4b6dbbbbe8613: Generated new TSK 0
I20260811 06:35:31.655375 31431 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:35:31.671846 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 31299
I20260811 06:35:31.673167 31392 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:35:31.794523 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 31299
W20260811 06:35:31.822937 31279 connection.cc:535] client connection to 127.28.239.254:38193 recv error: Network error: failed to read from TLS socket (remote: 127.28.239.254:38193): Connection reset by peer (error 104)
2026-08-11T06:35:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2518 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786429755927508-29631-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:35:31 dist-test-slave-gkw7 krb5kdc[31459](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:35:31 dist-test-slave-gkw7 krb5kdc[31459](info): set up 2 sockets
Aug 11 06:35:31 dist-test-slave-gkw7 krb5kdc[31459](info): commencing operation
krb5kdc: starting...
W20260811 06:35:33.175272 29631 mini_kdc.cc:121] Time spent starting KDC: real 1.333s user 0.000s sys 0.054s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786429755927508-29631-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786429755927508-29631-0/krb5kdc/test-user.keytab.
Aug 11 06:35:33 dist-test-slave-gkw7 krb5kdc[31459](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430133, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:35:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:35:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.239.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.239.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.239.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab.
Entry for principal kudu/127.28.239.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab.
I20260811 06:35:33.537252 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786429755927508-29631-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786429755927508-29631-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786429755927508-29631-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:42875
--webserver_interface=127.28.239.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab
--principal=kudu/127.28.239.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:45939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.239.254:42875
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:35:33.906518 31478 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:35:33.906816 31478 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:35:33.906973 31478 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:35:33.914317 31478 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:35:33.914427 31478 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:35:33.914530 31478 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:35:33.914637 31478 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:35:33.914726 31478 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:35:33.924768 31478 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:45939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786429755927508-29631-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786429755927508-29631-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.239.254:42875
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab
--principal=kudu/127.28.239.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:42875
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.239.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786429755927508-29631-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:35:33.926203 31478 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:35:33.927598 31478 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:35:33.939956 31483 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:35:33.939992 31484 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:35:33.940246 31486 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:35:33.941051 31478 server_base.cc:1034] running on GCE node
I20260811 06:35:33.941988 31478 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:35:33.943555 31478 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:35:33.944828 31478 hybrid_clock.cc:648] HybridClock initialized: now 1786430133944785 us; error 46 us; skew 500 ppm
I20260811 06:35:33.945300 31478 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:35:33 dist-test-slave-gkw7 krb5kdc[31459](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430133, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:35:33.948853 31478 init.cc:377] Logged in from keytab as kudu/127.28.239.254@KRBTEST.COM (short username kudu)
I20260811 06:35:33.950598 31478 webserver.cc:460] Webserver started at http://127.28.239.254:38099/ using document root <none> and password file <none>
I20260811 06:35:33.951382 31478 fs_manager.cc:359] Metadata directory not provided
I20260811 06:35:33.951550 31478 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:35:33.951889 31478 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:35:33.955166 31478 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786429755927508-29631-0/minicluster-data/master-0/data/instance:
uuid: "31112dd570684dfbafd9417fc031eee8"
format_stamp: "Formatted at 2026-08-11 06:35:33 on dist-test-slave-gkw7"
server_key: "cdcdc4d73d81122d011f66e475235c34"
server_key_iv: "c9062c5526cea2f9be1fc719b76d673d"
server_key_version: "encryptionkey@0"
I20260811 06:35:33.955973 31478 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786429755927508-29631-0/minicluster-data/master-0/wal/instance:
uuid: "31112dd570684dfbafd9417fc031eee8"
format_stamp: "Formatted at 2026-08-11 06:35:33 on dist-test-slave-gkw7"
server_key: "cdcdc4d73d81122d011f66e475235c34"
server_key_iv: "c9062c5526cea2f9be1fc719b76d673d"
server_key_version: "encryptionkey@0"
I20260811 06:35:33.961656 31478 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260811 06:35:33.965350 31496 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:35:33.967173 31478 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:35:33.967353 31478 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786429755927508-29631-0/minicluster-data/master-0/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786429755927508-29631-0/minicluster-data/master-0/wal
uuid: "31112dd570684dfbafd9417fc031eee8"
format_stamp: "Formatted at 2026-08-11 06:35:33 on dist-test-slave-gkw7"
server_key: "cdcdc4d73d81122d011f66e475235c34"
server_key_iv: "c9062c5526cea2f9be1fc719b76d673d"
server_key_version: "encryptionkey@0"
I20260811 06:35:33.967612 31478 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786429755927508-29631-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786429755927508-29631-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786429755927508-29631-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:35:33.985020 31478 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:35:33.989650 31478 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:35:33.989998 31478 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:35:34.010748 31478 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.254:42875
I20260811 06:35:34.010782 31585 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.254:42875 every 8 connection(s)
I20260811 06:35:34.012392 31478 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
I20260811 06:35:34.013360 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 31478
I20260811 06:35:34.013638 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786429755927508-29631-0/minicluster-data/master-0/wal/instance
I20260811 06:35:34.014078 29631 external_mini_cluster.cc:1428] Setting key e7e7eefd17ab38072b354cce5f09761e
I20260811 06:35:34.017397 31586 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:35:34.030627 31591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:34.016010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50149 (local address 127.28.239.254:42875)
0811 06:35:34.016852 (+ 842us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:34.016869 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:34.016923 (+ 54us) server_negotiation.cc:407] Connection header received
0811 06:35:34.019319 (+ 2396us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:34.019357 (+ 38us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:34.019932 (+ 575us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:34.020406 (+ 474us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:34.021003 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:34.023143 (+ 2140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:34.024674 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:34.025806 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:34.026011 (+ 205us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:34.027465 (+ 1454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:35:34.027478 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:35:34.027505 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:35:34.028324 (+ 819us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:35:34.028436 (+ 112us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:35:34.028483 (+ 47us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:35:34.028650 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0811 06:35:34.028997 (+ 347us) server_negotiation.cc:299] Negotiation successful
0811 06:35:34.029203 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":583,"thread_start_us":326,"threads_started":1}
I20260811 06:35:34.032059 31586 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31112dd570684dfbafd9417fc031eee8: Bootstrap starting.
I20260811 06:35:34.034822 29631 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 06:35:34.035616 31586 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 31112dd570684dfbafd9417fc031eee8: Neither blocks nor log segments found. Creating new log.
I20260811 06:35:34.036893 31586 log.cc:824] T 00000000000000000000000000000000 P 31112dd570684dfbafd9417fc031eee8: Log is configured to *not* fsync() on all Append() calls
I20260811 06:35:34.040014 31586 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31112dd570684dfbafd9417fc031eee8: No bootstrap required, opened a new log
I20260811 06:35:34.047782 31591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:34.040476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33463 (local address 127.28.239.254:42875)
0811 06:35:34.040708 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:34.040714 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:34.041093 (+ 379us) server_negotiation.cc:407] Connection header received
0811 06:35:34.041266 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:34.041272 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:34.041353 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:34.041482 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:34.042065 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:34.043405 (+ 1340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:34.044725 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:34.045485 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:34.045674 (+ 189us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:34.046881 (+ 1207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:35:34.046889 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:35:34.046894 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:35:34.046919 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:35:34.046970 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:35:34.046998 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:35:34.047137 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 06:35:34.047304 (+ 167us) server_negotiation.cc:299] Negotiation successful
0811 06:35:34.047378 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 06:35:34.051489 31586 raft_consensus.cc:348] T 00000000000000000000000000000000 P 31112dd570684dfbafd9417fc031eee8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31112dd570684dfbafd9417fc031eee8" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 42875 } }
I20260811 06:35:34.051941 31586 raft_consensus.cc:374] T 00000000000000000000000000000000 P 31112dd570684dfbafd9417fc031eee8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:35:34.052094 31586 raft_consensus.cc:729] T 00000000000000000000000000000000 P 31112dd570684dfbafd9417fc031eee8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31112dd570684dfbafd9417fc031eee8, State: Initialized, Role: FOLLOWER
I20260811 06:35:34.052769 31586 consensus_queue.cc:260] T 00000000000000000000000000000000 P 31112dd570684dfbafd9417fc031eee8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31112dd570684dfbafd9417fc031eee8" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 42875 } }
I20260811 06:35:34.053035 31586 raft_consensus.cc:388] T 00000000000000000000000000000000 P 31112dd570684dfbafd9417fc031eee8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:35:34.053228 31586 raft_consensus.cc:482] T 00000000000000000000000000000000 P 31112dd570684dfbafd9417fc031eee8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:35:34.053426 31586 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 31112dd570684dfbafd9417fc031eee8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:35:34.056512 31586 raft_consensus.cc:504] T 00000000000000000000000000000000 P 31112dd570684dfbafd9417fc031eee8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31112dd570684dfbafd9417fc031eee8" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 42875 } }
I20260811 06:35:34.057150 31586 leader_election.cc:302] T 00000000000000000000000000000000 P 31112dd570684dfbafd9417fc031eee8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 31112dd570684dfbafd9417fc031eee8; no voters:
I20260811 06:35:34.057919 31586 leader_election.cc:288] T 00000000000000000000000000000000 P 31112dd570684dfbafd9417fc031eee8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:35:34.058092 31602 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 31112dd570684dfbafd9417fc031eee8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:35:34.059614 31602 raft_consensus.cc:686] T 00000000000000000000000000000000 P 31112dd570684dfbafd9417fc031eee8 [term 1 LEADER]: Becoming Leader. State: Replica: 31112dd570684dfbafd9417fc031eee8, State: Running, Role: LEADER
I20260811 06:35:34.060268 31602 consensus_queue.cc:237] T 00000000000000000000000000000000 P 31112dd570684dfbafd9417fc031eee8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31112dd570684dfbafd9417fc031eee8" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 42875 } }
I20260811 06:35:34.060474 31586 sys_catalog.cc:564] T 00000000000000000000000000000000 P 31112dd570684dfbafd9417fc031eee8 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:35:34.063685 31604 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31112dd570684dfbafd9417fc031eee8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "31112dd570684dfbafd9417fc031eee8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31112dd570684dfbafd9417fc031eee8" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 42875 } } }
I20260811 06:35:34.063680 31605 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31112dd570684dfbafd9417fc031eee8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 31112dd570684dfbafd9417fc031eee8. Latest consensus state: current_term: 1 leader_uuid: "31112dd570684dfbafd9417fc031eee8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31112dd570684dfbafd9417fc031eee8" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 42875 } } }
I20260811 06:35:34.064224 31604 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31112dd570684dfbafd9417fc031eee8 [sys.catalog]: This master's current role is: LEADER
I20260811 06:35:34.064257 31605 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31112dd570684dfbafd9417fc031eee8 [sys.catalog]: This master's current role is: LEADER
I20260811 06:35:34.071118 31614 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:35:34.077217 31614 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:35:34.087267 31614 catalog_manager.cc:1347] Generated new cluster ID: e5bbf14c48034da6981c9dc6011a072f
I20260811 06:35:34.087420 31614 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:35:34.116026 31614 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:35:34.118269 31614 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:35:34.152137 31614 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 31112dd570684dfbafd9417fc031eee8: Generated new TSK 0
I20260811 06:35:34.153307 31614 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:35:34.168658 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 31478
W20260811 06:35:34.170221 31578 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:50149
W20260811 06:35:34.171186 29631 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 31478: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 06:35:34.171348 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 31478
W20260811 06:35:34.200047 31457 connection.cc:535] client connection to 127.28.239.254:42875 recv error: Network error: failed to read from TLS socket (remote: 127.28.239.254:42875): Connection reset by peer (error 104)
2026-08-11T06:35:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (2376 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786429755927508-29631-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:35:34 dist-test-slave-gkw7 krb5kdc[31632](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:35:34 dist-test-slave-gkw7 krb5kdc[31632](info): set up 2 sockets
Aug 11 06:35:34 dist-test-slave-gkw7 krb5kdc[31632](info): commencing operation
krb5kdc: starting...
W20260811 06:35:35.544620 29631 mini_kdc.cc:121] Time spent starting KDC: real 1.325s user 0.000s sys 0.052s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786429755927508-29631-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786429755927508-29631-0/krb5kdc/test-user.keytab.
Aug 11 06:35:35 dist-test-slave-gkw7 krb5kdc[31632](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430135, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:35:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:35:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.239.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.239.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.239.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab.
Entry for principal kudu/127.28.239.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab.
I20260811 06:35:35.897331 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786429755927508-29631-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786429755927508-29631-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786429755927508-29631-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:33677
--webserver_interface=127.28.239.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab
--principal=kudu/127.28.239.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:39009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.239.254:33677
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:35:36.258782 31649 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:35:36.259091 31649 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:35:36.259196 31649 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:35:36.266399 31649 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:35:36.266515 31649 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:35:36.266562 31649 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:35:36.266626 31649 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:35:36.266664 31649 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:35:36.276820 31649 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:39009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786429755927508-29631-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786429755927508-29631-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.239.254:33677
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab
--principal=kudu/127.28.239.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:33677
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.239.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786429755927508-29631-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:35:36.278203 31649 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:35:36.279618 31649 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:35:36.292791 31660 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:35:36.292791 31664 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:35:36.292816 31662 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:35:36.293843 31649 server_base.cc:1034] running on GCE node
I20260811 06:35:36.294726 31649 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:35:36.296394 31649 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:35:36.297662 31649 hybrid_clock.cc:648] HybridClock initialized: now 1786430136297583 us; error 77 us; skew 500 ppm
I20260811 06:35:36.298094 31649 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:35:36 dist-test-slave-gkw7 krb5kdc[31632](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430136, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:35:36.301723 31649 init.cc:377] Logged in from keytab as kudu/127.28.239.254@KRBTEST.COM (short username kudu)
I20260811 06:35:36.303442 31649 webserver.cc:460] Webserver started at http://127.28.239.254:43533/ using document root <none> and password file <none>
I20260811 06:35:36.304176 31649 fs_manager.cc:359] Metadata directory not provided
I20260811 06:35:36.304296 31649 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:35:36.304559 31649 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:35:36.307787 31649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786429755927508-29631-0/minicluster-data/master-0/data/instance:
uuid: "aed9d163cc2f4f81a5253c92bf5e5198"
format_stamp: "Formatted at 2026-08-11 06:35:36 on dist-test-slave-gkw7"
server_key: "69e8fb6e845360518cad9e5dfdd13575"
server_key_iv: "0cf1b1967af1a98b406cca89dafe4948"
server_key_version: "encryptionkey@0"
I20260811 06:35:36.308539 31649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786429755927508-29631-0/minicluster-data/master-0/wal/instance:
uuid: "aed9d163cc2f4f81a5253c92bf5e5198"
format_stamp: "Formatted at 2026-08-11 06:35:36 on dist-test-slave-gkw7"
server_key: "69e8fb6e845360518cad9e5dfdd13575"
server_key_iv: "0cf1b1967af1a98b406cca89dafe4948"
server_key_version: "encryptionkey@0"
I20260811 06:35:36.314195 31649 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 06:35:36.318055 31678 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:35:36.319579 31649 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 06:35:36.319797 31649 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786429755927508-29631-0/minicluster-data/master-0/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786429755927508-29631-0/minicluster-data/master-0/wal
uuid: "aed9d163cc2f4f81a5253c92bf5e5198"
format_stamp: "Formatted at 2026-08-11 06:35:36 on dist-test-slave-gkw7"
server_key: "69e8fb6e845360518cad9e5dfdd13575"
server_key_iv: "0cf1b1967af1a98b406cca89dafe4948"
server_key_version: "encryptionkey@0"
I20260811 06:35:36.320039 31649 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786429755927508-29631-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786429755927508-29631-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786429755927508-29631-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:35:36.326287 31649 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:35:36.330734 31649 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:35:36.331067 31649 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:35:36.351068 31649 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.254:33677
I20260811 06:35:36.351109 31763 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.254:33677 every 8 connection(s)
I20260811 06:35:36.352540 31649 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
I20260811 06:35:36.352821 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 31649
I20260811 06:35:36.353006 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786429755927508-29631-0/minicluster-data/master-0/wal/instance
I20260811 06:35:36.353487 29631 external_mini_cluster.cc:1428] Setting key 43c2d144ae794a7ba687b477d7fb1f5f
I20260811 06:35:36.357775 31765 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:35:36.362983 31772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:36.355621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34829 (local address 127.28.239.254:33677)
0811 06:35:36.356458 (+ 837us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:36.356474 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:36.356526 (+ 52us) server_negotiation.cc:407] Connection header received
0811 06:35:36.358952 (+ 2426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:36.358984 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:36.359429 (+ 445us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:36.359891 (+ 462us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:36.360919 (+ 1028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:35:36.360928 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:35:36.360946 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:35:36.360986 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:35:36.361081 (+ 95us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:35:36.361114 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:35:36.361337 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0811 06:35:36.361478 (+ 141us) server_negotiation.cc:299] Negotiation successful
0811 06:35:36.361720 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":569,"thread_start_us":335,"threads_started":1}
I20260811 06:35:36.367199 29631 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 06:35:36.371155 31765 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aed9d163cc2f4f81a5253c92bf5e5198: Bootstrap starting.
I20260811 06:35:36.374449 31772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:36.371838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58719 (local address 127.28.239.254:33677)
0811 06:35:36.372013 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:36.372019 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:36.372445 (+ 426us) server_negotiation.cc:407] Connection header received
0811 06:35:36.372580 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:36.372585 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:36.372652 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:36.372790 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:36.373785 (+ 995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:35:36.373794 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:35:36.373801 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:35:36.373822 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:35:36.373868 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:35:36.373891 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:35:36.374011 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 06:35:36.374097 (+ 86us) server_negotiation.cc:299] Negotiation successful
0811 06:35:36.374163 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 06:35:36.375195 31765 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aed9d163cc2f4f81a5253c92bf5e5198: Neither blocks nor log segments found. Creating new log.
I20260811 06:35:36.376564 31765 log.cc:824] T 00000000000000000000000000000000 P aed9d163cc2f4f81a5253c92bf5e5198: Log is configured to *not* fsync() on all Append() calls
I20260811 06:35:36.379906 31765 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aed9d163cc2f4f81a5253c92bf5e5198: No bootstrap required, opened a new log
I20260811 06:35:36.388815 31765 raft_consensus.cc:348] T 00000000000000000000000000000000 P aed9d163cc2f4f81a5253c92bf5e5198 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aed9d163cc2f4f81a5253c92bf5e5198" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 33677 } }
I20260811 06:35:36.389298 31765 raft_consensus.cc:374] T 00000000000000000000000000000000 P aed9d163cc2f4f81a5253c92bf5e5198 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:35:36.389391 31765 raft_consensus.cc:729] T 00000000000000000000000000000000 P aed9d163cc2f4f81a5253c92bf5e5198 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aed9d163cc2f4f81a5253c92bf5e5198, State: Initialized, Role: FOLLOWER
I20260811 06:35:36.390013 31765 consensus_queue.cc:260] T 00000000000000000000000000000000 P aed9d163cc2f4f81a5253c92bf5e5198 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aed9d163cc2f4f81a5253c92bf5e5198" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 33677 } }
I20260811 06:35:36.390250 31765 raft_consensus.cc:388] T 00000000000000000000000000000000 P aed9d163cc2f4f81a5253c92bf5e5198 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:35:36.390360 31765 raft_consensus.cc:482] T 00000000000000000000000000000000 P aed9d163cc2f4f81a5253c92bf5e5198 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:35:36.390506 31765 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aed9d163cc2f4f81a5253c92bf5e5198 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:35:36.392817 31765 raft_consensus.cc:504] T 00000000000000000000000000000000 P aed9d163cc2f4f81a5253c92bf5e5198 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aed9d163cc2f4f81a5253c92bf5e5198" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 33677 } }
I20260811 06:35:36.393330 31765 leader_election.cc:302] T 00000000000000000000000000000000 P aed9d163cc2f4f81a5253c92bf5e5198 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aed9d163cc2f4f81a5253c92bf5e5198; no voters:
I20260811 06:35:36.394029 31765 leader_election.cc:288] T 00000000000000000000000000000000 P aed9d163cc2f4f81a5253c92bf5e5198 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:35:36.394209 31782 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aed9d163cc2f4f81a5253c92bf5e5198 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:35:36.394925 31782 raft_consensus.cc:686] T 00000000000000000000000000000000 P aed9d163cc2f4f81a5253c92bf5e5198 [term 1 LEADER]: Becoming Leader. State: Replica: aed9d163cc2f4f81a5253c92bf5e5198, State: Running, Role: LEADER
I20260811 06:35:36.395617 31782 consensus_queue.cc:237] T 00000000000000000000000000000000 P aed9d163cc2f4f81a5253c92bf5e5198 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aed9d163cc2f4f81a5253c92bf5e5198" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 33677 } }
I20260811 06:35:36.396456 31765 sys_catalog.cc:564] T 00000000000000000000000000000000 P aed9d163cc2f4f81a5253c92bf5e5198 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:35:36.398826 31785 sys_catalog.cc:455] T 00000000000000000000000000000000 P aed9d163cc2f4f81a5253c92bf5e5198 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aed9d163cc2f4f81a5253c92bf5e5198. Latest consensus state: current_term: 1 leader_uuid: "aed9d163cc2f4f81a5253c92bf5e5198" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aed9d163cc2f4f81a5253c92bf5e5198" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 33677 } } }
I20260811 06:35:36.399101 31783 sys_catalog.cc:455] T 00000000000000000000000000000000 P aed9d163cc2f4f81a5253c92bf5e5198 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aed9d163cc2f4f81a5253c92bf5e5198" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aed9d163cc2f4f81a5253c92bf5e5198" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 33677 } } }
I20260811 06:35:36.399386 31785 sys_catalog.cc:458] T 00000000000000000000000000000000 P aed9d163cc2f4f81a5253c92bf5e5198 [sys.catalog]: This master's current role is: LEADER
I20260811 06:35:36.399396 31783 sys_catalog.cc:458] T 00000000000000000000000000000000 P aed9d163cc2f4f81a5253c92bf5e5198 [sys.catalog]: This master's current role is: LEADER
I20260811 06:35:36.402534 31796 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:35:36.408217 31796 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:35:36.418044 31796 catalog_manager.cc:1347] Generated new cluster ID: 2352ec98aeb24bb8a8fbe8b85d4615c1
I20260811 06:35:36.418167 31796 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:35:36.452992 31796 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:35:36.453519 31796 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:35:36.461352 31796 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aed9d163cc2f4f81a5253c92bf5e5198: Generated new TSK 0
I20260811 06:35:36.462297 31796 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:35:36.481432 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 31649
W20260811 06:35:36.482728 31758 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:34829
W20260811 06:35:36.483434 29631 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 31649: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 06:35:36.483590 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 31649
2026-08-11T06:35:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (2310 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786429755927508-29631-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:35:36 dist-test-slave-gkw7 krb5kdc[31822](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:35:36 dist-test-slave-gkw7 krb5kdc[31822](info): set up 2 sockets
Aug 11 06:35:36 dist-test-slave-gkw7 krb5kdc[31822](info): commencing operation
krb5kdc: starting...
W20260811 06:35:37.858021 29631 mini_kdc.cc:121] Time spent starting KDC: real 1.328s user 0.005s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786429755927508-29631-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786429755927508-29631-0/krb5kdc/test-user.keytab.
Aug 11 06:35:38 dist-test-slave-gkw7 krb5kdc[31822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430138, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:35:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:35:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.239.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.239.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.239.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab.
Entry for principal kudu/127.28.239.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab.
I20260811 06:35:38.229954 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786429755927508-29631-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786429755927508-29631-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786429755927508-29631-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:37147
--webserver_interface=127.28.239.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab
--principal=kudu/127.28.239.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:34007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.239.254:37147
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:35:38.592562 31840 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:35:38.592864 31840 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:35:38.593032 31840 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:35:38.600455 31840 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:35:38.600569 31840 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:35:38.600665 31840 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:35:38.600778 31840 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:35:38.600869 31840 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:35:38.610920 31840 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:34007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786429755927508-29631-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786429755927508-29631-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.239.254:37147
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab
--principal=kudu/127.28.239.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:37147
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.239.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786429755927508-29631-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:35:38.612332 31840 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:35:38.613865 31840 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:35:38.626539 31849 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:35:38.626583 31853 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:35:38.626626 31850 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:35:38.627946 31840 server_base.cc:1034] running on GCE node
I20260811 06:35:38.628844 31840 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:35:38.630386 31840 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:35:38.631636 31840 hybrid_clock.cc:648] HybridClock initialized: now 1786430138631565 us; error 71 us; skew 500 ppm
I20260811 06:35:38.632019 31840 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:35:38 dist-test-slave-gkw7 krb5kdc[31822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430138, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:35:38.635389 31840 init.cc:377] Logged in from keytab as kudu/127.28.239.254@KRBTEST.COM (short username kudu)
I20260811 06:35:38.636942 31840 webserver.cc:460] Webserver started at http://127.28.239.254:46151/ using document root <none> and password file <none>
I20260811 06:35:38.637742 31840 fs_manager.cc:359] Metadata directory not provided
I20260811 06:35:38.637902 31840 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:35:38.638230 31840 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:35:38.641332 31840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786429755927508-29631-0/minicluster-data/master-0/data/instance:
uuid: "139d6bc1004e4e98983d3e4b031a6b0c"
format_stamp: "Formatted at 2026-08-11 06:35:38 on dist-test-slave-gkw7"
server_key: "d0466748c7cdaf34744af10fbf3bd25d"
server_key_iv: "c1352efba7ccd03ccb1d58ded8a39bf6"
server_key_version: "encryptionkey@0"
I20260811 06:35:38.642220 31840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786429755927508-29631-0/minicluster-data/master-0/wal/instance:
uuid: "139d6bc1004e4e98983d3e4b031a6b0c"
format_stamp: "Formatted at 2026-08-11 06:35:38 on dist-test-slave-gkw7"
server_key: "d0466748c7cdaf34744af10fbf3bd25d"
server_key_iv: "c1352efba7ccd03ccb1d58ded8a39bf6"
server_key_version: "encryptionkey@0"
I20260811 06:35:38.647502 31840 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 06:35:38.651172 31863 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:35:38.652529 31840 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260811 06:35:38.652745 31840 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786429755927508-29631-0/minicluster-data/master-0/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786429755927508-29631-0/minicluster-data/master-0/wal
uuid: "139d6bc1004e4e98983d3e4b031a6b0c"
format_stamp: "Formatted at 2026-08-11 06:35:38 on dist-test-slave-gkw7"
server_key: "d0466748c7cdaf34744af10fbf3bd25d"
server_key_iv: "c1352efba7ccd03ccb1d58ded8a39bf6"
server_key_version: "encryptionkey@0"
I20260811 06:35:38.652964 31840 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786429755927508-29631-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786429755927508-29631-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786429755927508-29631-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:35:38.679332 31840 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:35:38.683554 31840 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:35:38.683892 31840 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:35:38.703522 31840 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.254:37147
I20260811 06:35:38.703542 31947 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.254:37147 every 8 connection(s)
I20260811 06:35:38.704975 31840 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
I20260811 06:35:38.706234 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 31840
I20260811 06:35:38.706420 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786429755927508-29631-0/minicluster-data/master-0/wal/instance
I20260811 06:35:38.706888 29631 external_mini_cluster.cc:1428] Setting key fa6c4d62ede7851e5e60db259511f877
I20260811 06:35:38.709803 31948 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 06:35:38 dist-test-slave-gkw7 krb5kdc[31822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430138, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:35:38.725243 31948 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 139d6bc1004e4e98983d3e4b031a6b0c: Bootstrap starting.
I20260811 06:35:38.729945 31948 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 139d6bc1004e4e98983d3e4b031a6b0c: Neither blocks nor log segments found. Creating new log.
I20260811 06:35:38.731256 31948 log.cc:824] T 00000000000000000000000000000000 P 139d6bc1004e4e98983d3e4b031a6b0c: Log is configured to *not* fsync() on all Append() calls
I20260811 06:35:38.734632 31951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:38.708855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44471 (local address 127.28.239.254:37147)
0811 06:35:38.709723 (+ 868us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:38.709740 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:38.709802 (+ 62us) server_negotiation.cc:407] Connection header received
0811 06:35:38.712143 (+ 2341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:38.712170 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:38.712668 (+ 498us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:38.713144 (+ 476us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:38.714709 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:38.716821 (+ 2112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:38.718203 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:38.719284 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:38.719479 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:38.725676 (+ 6197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:35:38.725689 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:35:38.725712 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:35:38.725771 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:35:38.730030 (+ 4259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:35:38.730829 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:35:38.730838 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:35:38.730847 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:35:38.731093 (+ 246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:35:38.731536 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:35:38.731540 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:35:38.731542 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:35:38.732384 (+ 842us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:35:38.732727 (+ 343us) server_negotiation.cc:1035] Waiting for connection context
0811 06:35:38.733125 (+ 398us) server_negotiation.cc:299] Negotiation successful
0811 06:35:38.733541 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":617,"thread_start_us":369,"threads_started":1}
I20260811 06:35:38.734833 31948 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 139d6bc1004e4e98983d3e4b031a6b0c: No bootstrap required, opened a new log
I20260811 06:35:38.738507 29631 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 06:35:38.745188 31948 raft_consensus.cc:348] T 00000000000000000000000000000000 P 139d6bc1004e4e98983d3e4b031a6b0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "139d6bc1004e4e98983d3e4b031a6b0c" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 37147 } }
I20260811 06:35:38.745810 31948 raft_consensus.cc:374] T 00000000000000000000000000000000 P 139d6bc1004e4e98983d3e4b031a6b0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:35:38.745927 31948 raft_consensus.cc:729] T 00000000000000000000000000000000 P 139d6bc1004e4e98983d3e4b031a6b0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 139d6bc1004e4e98983d3e4b031a6b0c, State: Initialized, Role: FOLLOWER
I20260811 06:35:38.746691 31948 consensus_queue.cc:260] T 00000000000000000000000000000000 P 139d6bc1004e4e98983d3e4b031a6b0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "139d6bc1004e4e98983d3e4b031a6b0c" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 37147 } }
I20260811 06:35:38.747000 31948 raft_consensus.cc:388] T 00000000000000000000000000000000 P 139d6bc1004e4e98983d3e4b031a6b0c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:35:38.747169 31948 raft_consensus.cc:482] T 00000000000000000000000000000000 P 139d6bc1004e4e98983d3e4b031a6b0c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:35:38.747370 31948 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 139d6bc1004e4e98983d3e4b031a6b0c [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:35:38.750602 31948 raft_consensus.cc:504] T 00000000000000000000000000000000 P 139d6bc1004e4e98983d3e4b031a6b0c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "139d6bc1004e4e98983d3e4b031a6b0c" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 37147 } }
I20260811 06:35:38.751264 31948 leader_election.cc:302] T 00000000000000000000000000000000 P 139d6bc1004e4e98983d3e4b031a6b0c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 139d6bc1004e4e98983d3e4b031a6b0c; no voters:
I20260811 06:35:38.752007 31948 leader_election.cc:288] T 00000000000000000000000000000000 P 139d6bc1004e4e98983d3e4b031a6b0c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:35:38.752126 31965 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 139d6bc1004e4e98983d3e4b031a6b0c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:35:38.753882 31965 raft_consensus.cc:686] T 00000000000000000000000000000000 P 139d6bc1004e4e98983d3e4b031a6b0c [term 1 LEADER]: Becoming Leader. State: Replica: 139d6bc1004e4e98983d3e4b031a6b0c, State: Running, Role: LEADER
I20260811 06:35:38.754540 31948 sys_catalog.cc:564] T 00000000000000000000000000000000 P 139d6bc1004e4e98983d3e4b031a6b0c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:35:38.755182 31965 consensus_queue.cc:237] T 00000000000000000000000000000000 P 139d6bc1004e4e98983d3e4b031a6b0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "139d6bc1004e4e98983d3e4b031a6b0c" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 37147 } }
I20260811 06:35:38.758590 31966 sys_catalog.cc:455] T 00000000000000000000000000000000 P 139d6bc1004e4e98983d3e4b031a6b0c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "139d6bc1004e4e98983d3e4b031a6b0c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "139d6bc1004e4e98983d3e4b031a6b0c" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 37147 } } }
I20260811 06:35:38.759008 31966 sys_catalog.cc:458] T 00000000000000000000000000000000 P 139d6bc1004e4e98983d3e4b031a6b0c [sys.catalog]: This master's current role is: LEADER
I20260811 06:35:38.762094 31951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:38.743981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53667 (local address 127.28.239.254:37147)
0811 06:35:38.744231 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:38.744240 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:38.744884 (+ 644us) server_negotiation.cc:407] Connection header received
0811 06:35:38.745091 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:38.745095 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:38.745159 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:38.745249 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:38.747398 (+ 2149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:38.748247 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:38.750052 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:38.750496 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:38.750597 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:38.753757 (+ 3160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:35:38.753764 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:35:38.753768 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:35:38.753808 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:35:38.758032 (+ 4224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:35:38.759042 (+ 1010us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:35:38.759047 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:35:38.759051 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:35:38.759299 (+ 248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:35:38.759845 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:35:38.759853 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:35:38.759859 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:35:38.760376 (+ 517us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:35:38.760764 (+ 388us) server_negotiation.cc:1035] Waiting for connection context
0811 06:35:38.761282 (+ 518us) server_negotiation.cc:299] Negotiation successful
0811 06:35:38.761621 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260811 06:35:38.764219 31967 sys_catalog.cc:455] T 00000000000000000000000000000000 P 139d6bc1004e4e98983d3e4b031a6b0c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 139d6bc1004e4e98983d3e4b031a6b0c. Latest consensus state: current_term: 1 leader_uuid: "139d6bc1004e4e98983d3e4b031a6b0c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "139d6bc1004e4e98983d3e4b031a6b0c" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 37147 } } }
I20260811 06:35:38.764546 31967 sys_catalog.cc:458] T 00000000000000000000000000000000 P 139d6bc1004e4e98983d3e4b031a6b0c [sys.catalog]: This master's current role is: LEADER
I20260811 06:35:38.770291 31976 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:35:38.775116 31976 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:35:38.783792 31976 catalog_manager.cc:1347] Generated new cluster ID: 12a009e213df46648ca960584c26ee10
I20260811 06:35:38.783922 31976 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:35:38.817241 31976 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:35:38.819262 31976 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:35:38.829650 31976 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 139d6bc1004e4e98983d3e4b031a6b0c: Generated new TSK 0
I20260811 06:35:38.830761 31976 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:35:38.839233 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 31840
I20260811 06:35:38.840574 31941 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:35:38.940276 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 31840
2026-08-11T06:35:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2457 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786429755927508-29631-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:35:39 dist-test-slave-gkw7 krb5kdc[32002](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:35:39 dist-test-slave-gkw7 krb5kdc[32002](info): set up 2 sockets
Aug 11 06:35:39 dist-test-slave-gkw7 krb5kdc[32002](info): commencing operation
krb5kdc: starting...
W20260811 06:35:40.382858 29631 mini_kdc.cc:121] Time spent starting KDC: real 1.396s user 0.007s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786429755927508-29631-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786429755927508-29631-0/krb5kdc/test-user.keytab.
Aug 11 06:35:40 dist-test-slave-gkw7 krb5kdc[32002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430140, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:35:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:35:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.239.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.239.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.239.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab.
Entry for principal kudu/127.28.239.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab.
I20260811 06:35:40.745024 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786429755927508-29631-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786429755927508-29631-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786429755927508-29631-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:40569
--webserver_interface=127.28.239.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab
--principal=kudu/127.28.239.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:39397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.239.254:40569
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:35:41.103047 32024 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:35:41.103420 32024 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:35:41.103609 32024 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:35:41.110894 32024 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:35:41.111073 32024 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:35:41.111191 32024 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:35:41.111311 32024 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:35:41.111405 32024 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:35:41.121461 32024 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:39397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786429755927508-29631-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786429755927508-29631-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.239.254:40569
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab
--principal=kudu/127.28.239.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:40569
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.239.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786429755927508-29631-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:35:41.122979 32024 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:35:41.124387 32024 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:35:41.136822 32031 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:35:41.136914 32032 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:35:41.136790 32034 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:35:41.138062 32024 server_base.cc:1034] running on GCE node
I20260811 06:35:41.138939 32024 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:35:41.140372 32024 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:35:41.141669 32024 hybrid_clock.cc:648] HybridClock initialized: now 1786430141141601 us; error 62 us; skew 500 ppm
I20260811 06:35:41.142088 32024 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:35:41 dist-test-slave-gkw7 krb5kdc[32002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430141, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:35:41.145805 32024 init.cc:377] Logged in from keytab as kudu/127.28.239.254@KRBTEST.COM (short username kudu)
I20260811 06:35:41.147357 32024 webserver.cc:460] Webserver started at http://127.28.239.254:42003/ using document root <none> and password file <none>
I20260811 06:35:41.148061 32024 fs_manager.cc:359] Metadata directory not provided
I20260811 06:35:41.148166 32024 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:35:41.148449 32024 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:35:41.151688 32024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786429755927508-29631-0/minicluster-data/master-0/data/instance:
uuid: "af57a0f21b4f4eaab7c48b375a9027b8"
format_stamp: "Formatted at 2026-08-11 06:35:41 on dist-test-slave-gkw7"
server_key: "28888c459b29ef044baa88d86fe7fd43"
server_key_iv: "9822d4473c01f0bd7f5b4a4459a4f3d8"
server_key_version: "encryptionkey@0"
I20260811 06:35:41.152431 32024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786429755927508-29631-0/minicluster-data/master-0/wal/instance:
uuid: "af57a0f21b4f4eaab7c48b375a9027b8"
format_stamp: "Formatted at 2026-08-11 06:35:41 on dist-test-slave-gkw7"
server_key: "28888c459b29ef044baa88d86fe7fd43"
server_key_iv: "9822d4473c01f0bd7f5b4a4459a4f3d8"
server_key_version: "encryptionkey@0"
I20260811 06:35:41.158025 32024 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260811 06:35:41.161947 32046 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:35:41.163368 32024 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 06:35:41.163548 32024 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786429755927508-29631-0/minicluster-data/master-0/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786429755927508-29631-0/minicluster-data/master-0/wal
uuid: "af57a0f21b4f4eaab7c48b375a9027b8"
format_stamp: "Formatted at 2026-08-11 06:35:41 on dist-test-slave-gkw7"
server_key: "28888c459b29ef044baa88d86fe7fd43"
server_key_iv: "9822d4473c01f0bd7f5b4a4459a4f3d8"
server_key_version: "encryptionkey@0"
I20260811 06:35:41.163753 32024 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786429755927508-29631-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786429755927508-29631-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786429755927508-29631-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:35:41.183884 32024 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:35:41.188503 32024 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:35:41.188829 32024 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:35:41.209344 32024 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.254:40569
I20260811 06:35:41.209364 32122 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.254:40569 every 8 connection(s)
I20260811 06:35:41.210877 32024 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
I20260811 06:35:41.215358 32123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:35:41.220881 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 32024
I20260811 06:35:41.221107 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786429755927508-29631-0/minicluster-data/master-0/wal/instance
I20260811 06:35:41.221834 29631 external_mini_cluster.cc:1428] Setting key 02a2a66fb103c52e6180a2f245cdd769
I20260811 06:35:41.229060 32123 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af57a0f21b4f4eaab7c48b375a9027b8: Bootstrap starting.
I20260811 06:35:41.232764 32123 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P af57a0f21b4f4eaab7c48b375a9027b8: Neither blocks nor log segments found. Creating new log.
I20260811 06:35:41.234306 32123 log.cc:824] T 00000000000000000000000000000000 P af57a0f21b4f4eaab7c48b375a9027b8: Log is configured to *not* fsync() on all Append() calls
I20260811 06:35:41.236330 32128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:41.223932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32799 (local address 127.28.239.254:40569)
0811 06:35:41.224836 (+ 904us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:41.224849 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:41.224889 (+ 40us) server_negotiation.cc:407] Connection header received
0811 06:35:41.227229 (+ 2340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:41.227256 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:41.227663 (+ 407us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:41.228073 (+ 410us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:41.228648 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:41.230171 (+ 1523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:41.231554 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:41.232231 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:41.232433 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:41.233893 (+ 1460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:35:41.233900 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:35:41.233915 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:35:41.234471 (+ 556us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:35:41.234598 (+ 127us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:35:41.234643 (+ 45us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:35:41.234778 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 06:35:41.235004 (+ 226us) server_negotiation.cc:299] Negotiation successful
0811 06:35:41.235330 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":666,"thread_start_us":434,"threads_started":1}
I20260811 06:35:41.237955 32123 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af57a0f21b4f4eaab7c48b375a9027b8: No bootstrap required, opened a new log
I20260811 06:35:41.240226 29631 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 06:35:41.248747 32123 raft_consensus.cc:348] T 00000000000000000000000000000000 P af57a0f21b4f4eaab7c48b375a9027b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af57a0f21b4f4eaab7c48b375a9027b8" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 40569 } }
I20260811 06:35:41.249284 32123 raft_consensus.cc:374] T 00000000000000000000000000000000 P af57a0f21b4f4eaab7c48b375a9027b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:35:41.249449 32123 raft_consensus.cc:729] T 00000000000000000000000000000000 P af57a0f21b4f4eaab7c48b375a9027b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af57a0f21b4f4eaab7c48b375a9027b8, State: Initialized, Role: FOLLOWER
I20260811 06:35:41.250224 32123 consensus_queue.cc:260] T 00000000000000000000000000000000 P af57a0f21b4f4eaab7c48b375a9027b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af57a0f21b4f4eaab7c48b375a9027b8" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 40569 } }
I20260811 06:35:41.250545 32123 raft_consensus.cc:388] T 00000000000000000000000000000000 P af57a0f21b4f4eaab7c48b375a9027b8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:35:41.250766 32123 raft_consensus.cc:482] T 00000000000000000000000000000000 P af57a0f21b4f4eaab7c48b375a9027b8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:35:41.250993 32123 raft_consensus.cc:3037] T 00000000000000000000000000000000 P af57a0f21b4f4eaab7c48b375a9027b8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:35:41.252303 32128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:41.245792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55587 (local address 127.28.239.254:40569)
0811 06:35:41.245995 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:41.246004 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:41.246478 (+ 474us) server_negotiation.cc:407] Connection header received
0811 06:35:41.246710 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:41.246714 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:41.246781 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:41.246900 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:41.247439 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:41.248197 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:41.249649 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:41.250202 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:41.250354 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:41.251568 (+ 1214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:35:41.251573 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:35:41.251576 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:35:41.251595 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:35:41.251637 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:35:41.251655 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:35:41.251781 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 06:35:41.251897 (+ 116us) server_negotiation.cc:299] Negotiation successful
0811 06:35:41.251999 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 06:35:41.254451 32123 raft_consensus.cc:504] T 00000000000000000000000000000000 P af57a0f21b4f4eaab7c48b375a9027b8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af57a0f21b4f4eaab7c48b375a9027b8" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 40569 } }
I20260811 06:35:41.255082 32123 leader_election.cc:302] T 00000000000000000000000000000000 P af57a0f21b4f4eaab7c48b375a9027b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: af57a0f21b4f4eaab7c48b375a9027b8; no voters:
I20260811 06:35:41.255756 32123 leader_election.cc:288] T 00000000000000000000000000000000 P af57a0f21b4f4eaab7c48b375a9027b8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:35:41.255873 32139 raft_consensus.cc:2781] T 00000000000000000000000000000000 P af57a0f21b4f4eaab7c48b375a9027b8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:35:41.256808 32139 raft_consensus.cc:686] T 00000000000000000000000000000000 P af57a0f21b4f4eaab7c48b375a9027b8 [term 1 LEADER]: Becoming Leader. State: Replica: af57a0f21b4f4eaab7c48b375a9027b8, State: Running, Role: LEADER
I20260811 06:35:41.257507 32139 consensus_queue.cc:237] T 00000000000000000000000000000000 P af57a0f21b4f4eaab7c48b375a9027b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af57a0f21b4f4eaab7c48b375a9027b8" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 40569 } }
I20260811 06:35:41.258584 32123 sys_catalog.cc:564] T 00000000000000000000000000000000 P af57a0f21b4f4eaab7c48b375a9027b8 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:35:41.260560 32141 sys_catalog.cc:455] T 00000000000000000000000000000000 P af57a0f21b4f4eaab7c48b375a9027b8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader af57a0f21b4f4eaab7c48b375a9027b8. Latest consensus state: current_term: 1 leader_uuid: "af57a0f21b4f4eaab7c48b375a9027b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af57a0f21b4f4eaab7c48b375a9027b8" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 40569 } } }
I20260811 06:35:41.260471 32140 sys_catalog.cc:455] T 00000000000000000000000000000000 P af57a0f21b4f4eaab7c48b375a9027b8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "af57a0f21b4f4eaab7c48b375a9027b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af57a0f21b4f4eaab7c48b375a9027b8" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 40569 } } }
I20260811 06:35:41.260870 32140 sys_catalog.cc:458] T 00000000000000000000000000000000 P af57a0f21b4f4eaab7c48b375a9027b8 [sys.catalog]: This master's current role is: LEADER
I20260811 06:35:41.260867 32141 sys_catalog.cc:458] T 00000000000000000000000000000000 P af57a0f21b4f4eaab7c48b375a9027b8 [sys.catalog]: This master's current role is: LEADER
I20260811 06:35:41.261924 32147 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:35:41.270385 32147 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:35:41.288337 32147 catalog_manager.cc:1347] Generated new cluster ID: fa90a9eaba1d482eb0efb499b3c0e0f1
I20260811 06:35:41.288504 32147 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:35:41.335887 32147 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:35:41.337270 32147 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:35:41.362941 32147 catalog_manager.cc:5868] T 00000000000000000000000000000000 P af57a0f21b4f4eaab7c48b375a9027b8: Generated new TSK 0
I20260811 06:35:41.363870 32147 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:35:41.381177 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 32024
W20260811 06:35:41.382416 32112 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:32799
W20260811 06:35:41.383221 29631 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 32024: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 06:35:41.383423 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 32024
2026-08-11T06:35:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2448 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786429755927508-29631-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:35:41 dist-test-slave-gkw7 krb5kdc[32170](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:35:41 dist-test-slave-gkw7 krb5kdc[32170](info): set up 2 sockets
Aug 11 06:35:41 dist-test-slave-gkw7 krb5kdc[32170](info): commencing operation
krb5kdc: starting...
W20260811 06:35:42.772450 29631 mini_kdc.cc:121] Time spent starting KDC: real 1.336s user 0.005s sys 0.049s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786429755927508-29631-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786429755927508-29631-0/krb5kdc/test-user.keytab.
Aug 11 06:35:43 dist-test-slave-gkw7 krb5kdc[32170](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430143, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:35:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:35:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.239.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.239.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.239.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab.
Entry for principal kudu/127.28.239.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab.
I20260811 06:35:43.137778 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786429755927508-29631-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786429755927508-29631-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786429755927508-29631-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:36765
--webserver_interface=127.28.239.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab
--principal=kudu/127.28.239.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:43779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.239.254:36765
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:35:43.505882 32191 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:35:43.506208 32191 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:35:43.506445 32191 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:35:43.514005 32191 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:35:43.514170 32191 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:35:43.514261 32191 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:35:43.514379 32191 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:35:43.514446 32191 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:35:43.524575 32191 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:43779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786429755927508-29631-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786429755927508-29631-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.239.254:36765
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab
--principal=kudu/127.28.239.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:36765
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.239.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786429755927508-29631-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:35:43.526024 32191 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:35:43.527447 32191 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:35:43.539971 32197 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:35:43.540060 32200 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:35:43.540016 32202 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:35:43.541168 32191 server_base.cc:1034] running on GCE node
I20260811 06:35:43.542100 32191 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:35:43.543605 32191 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:35:43.544817 32191 hybrid_clock.cc:648] HybridClock initialized: now 1786430143544777 us; error 39 us; skew 500 ppm
I20260811 06:35:43.545194 32191 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:35:43 dist-test-slave-gkw7 krb5kdc[32170](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430143, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:35:43.548556 32191 init.cc:377] Logged in from keytab as kudu/127.28.239.254@KRBTEST.COM (short username kudu)
I20260811 06:35:43.550246 32191 webserver.cc:460] Webserver started at http://127.28.239.254:45049/ using document root <none> and password file <none>
I20260811 06:35:43.550915 32191 fs_manager.cc:359] Metadata directory not provided
I20260811 06:35:43.551019 32191 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:35:43.551297 32191 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:35:43.554867 32191 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786429755927508-29631-0/minicluster-data/master-0/data/instance:
uuid: "96862a30785b4cdb9d0c2deba5d6da5a"
format_stamp: "Formatted at 2026-08-11 06:35:43 on dist-test-slave-gkw7"
server_key: "05043f8cfd8acef15c82890cc782c2da"
server_key_iv: "34594674984052d1cb9bc7dc35279b37"
server_key_version: "encryptionkey@0"
I20260811 06:35:43.555605 32191 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786429755927508-29631-0/minicluster-data/master-0/wal/instance:
uuid: "96862a30785b4cdb9d0c2deba5d6da5a"
format_stamp: "Formatted at 2026-08-11 06:35:43 on dist-test-slave-gkw7"
server_key: "05043f8cfd8acef15c82890cc782c2da"
server_key_iv: "34594674984052d1cb9bc7dc35279b37"
server_key_version: "encryptionkey@0"
I20260811 06:35:43.560952 32191 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 06:35:43.564767 32212 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:35:43.566244 32191 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:35:43.566392 32191 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786429755927508-29631-0/minicluster-data/master-0/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786429755927508-29631-0/minicluster-data/master-0/wal
uuid: "96862a30785b4cdb9d0c2deba5d6da5a"
format_stamp: "Formatted at 2026-08-11 06:35:43 on dist-test-slave-gkw7"
server_key: "05043f8cfd8acef15c82890cc782c2da"
server_key_iv: "34594674984052d1cb9bc7dc35279b37"
server_key_version: "encryptionkey@0"
I20260811 06:35:43.566560 32191 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786429755927508-29631-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786429755927508-29631-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786429755927508-29631-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:35:43.572536 32191 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:35:43.576823 32191 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:35:43.577077 32191 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:35:43.596078 32191 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.254:36765
I20260811 06:35:43.596117 32286 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.254:36765 every 8 connection(s)
I20260811 06:35:43.597551 32191 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
I20260811 06:35:43.601331 32288 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:35:43.602825 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 32191
I20260811 06:35:43.603008 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786429755927508-29631-0/minicluster-data/master-0/wal/instance
I20260811 06:35:43.603564 29631 external_mini_cluster.cc:1428] Setting key 2f2e15a6d7a0e4db76a8a326eda8e8f0
I20260811 06:35:43.611644 32291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:43.605508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51211 (local address 127.28.239.254:36765)
0811 06:35:43.606320 (+ 812us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:43.606332 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:43.606377 (+ 45us) server_negotiation.cc:407] Connection header received
0811 06:35:43.608000 (+ 1623us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:43.608022 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:43.608455 (+ 433us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:43.608872 (+ 417us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:43.609935 (+ 1063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:35:43.609942 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:35:43.609955 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:35:43.609969 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:35:43.610025 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:35:43.610047 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:35:43.610296 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0811 06:35:43.610443 (+ 147us) server_negotiation.cc:299] Negotiation successful
0811 06:35:43.610732 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":547,"thread_start_us":328,"threads_started":1}
I20260811 06:35:43.615458 32288 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96862a30785b4cdb9d0c2deba5d6da5a: Bootstrap starting.
I20260811 06:35:43.615620 29631 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 06:35:43.618852 32288 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 96862a30785b4cdb9d0c2deba5d6da5a: Neither blocks nor log segments found. Creating new log.
I20260811 06:35:43.620159 32288 log.cc:824] T 00000000000000000000000000000000 P 96862a30785b4cdb9d0c2deba5d6da5a: Log is configured to *not* fsync() on all Append() calls
I20260811 06:35:43.622871 32291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:43.620067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55153 (local address 127.28.239.254:36765)
0811 06:35:43.620261 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:43.620269 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:43.620729 (+ 460us) server_negotiation.cc:407] Connection header received
0811 06:35:43.620933 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:43.620937 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:43.620987 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:43.621102 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:43.622140 (+ 1038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:35:43.622146 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:35:43.622150 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:35:43.622168 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:35:43.622213 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:35:43.622234 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:35:43.622359 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 06:35:43.622514 (+ 155us) server_negotiation.cc:299] Negotiation successful
0811 06:35:43.622576 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 06:35:43.623647 32288 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96862a30785b4cdb9d0c2deba5d6da5a: No bootstrap required, opened a new log
I20260811 06:35:43.631111 32288 raft_consensus.cc:348] T 00000000000000000000000000000000 P 96862a30785b4cdb9d0c2deba5d6da5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96862a30785b4cdb9d0c2deba5d6da5a" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 36765 } }
I20260811 06:35:43.631534 32288 raft_consensus.cc:374] T 00000000000000000000000000000000 P 96862a30785b4cdb9d0c2deba5d6da5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:35:43.631652 32288 raft_consensus.cc:729] T 00000000000000000000000000000000 P 96862a30785b4cdb9d0c2deba5d6da5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96862a30785b4cdb9d0c2deba5d6da5a, State: Initialized, Role: FOLLOWER
I20260811 06:35:43.632344 32288 consensus_queue.cc:260] T 00000000000000000000000000000000 P 96862a30785b4cdb9d0c2deba5d6da5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96862a30785b4cdb9d0c2deba5d6da5a" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 36765 } }
I20260811 06:35:43.632671 32288 raft_consensus.cc:388] T 00000000000000000000000000000000 P 96862a30785b4cdb9d0c2deba5d6da5a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:35:43.632834 32288 raft_consensus.cc:482] T 00000000000000000000000000000000 P 96862a30785b4cdb9d0c2deba5d6da5a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:35:43.633033 32288 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 96862a30785b4cdb9d0c2deba5d6da5a [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:35:43.635356 32288 raft_consensus.cc:504] T 00000000000000000000000000000000 P 96862a30785b4cdb9d0c2deba5d6da5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96862a30785b4cdb9d0c2deba5d6da5a" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 36765 } }
I20260811 06:35:43.635861 32288 leader_election.cc:302] T 00000000000000000000000000000000 P 96862a30785b4cdb9d0c2deba5d6da5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 96862a30785b4cdb9d0c2deba5d6da5a; no voters:
I20260811 06:35:43.636571 32288 leader_election.cc:288] T 00000000000000000000000000000000 P 96862a30785b4cdb9d0c2deba5d6da5a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:35:43.636763 32304 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 96862a30785b4cdb9d0c2deba5d6da5a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:35:43.637482 32304 raft_consensus.cc:686] T 00000000000000000000000000000000 P 96862a30785b4cdb9d0c2deba5d6da5a [term 1 LEADER]: Becoming Leader. State: Replica: 96862a30785b4cdb9d0c2deba5d6da5a, State: Running, Role: LEADER
I20260811 06:35:43.638139 32304 consensus_queue.cc:237] T 00000000000000000000000000000000 P 96862a30785b4cdb9d0c2deba5d6da5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96862a30785b4cdb9d0c2deba5d6da5a" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 36765 } }
I20260811 06:35:43.638943 32288 sys_catalog.cc:564] T 00000000000000000000000000000000 P 96862a30785b4cdb9d0c2deba5d6da5a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:35:43.641718 32307 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96862a30785b4cdb9d0c2deba5d6da5a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 96862a30785b4cdb9d0c2deba5d6da5a. Latest consensus state: current_term: 1 leader_uuid: "96862a30785b4cdb9d0c2deba5d6da5a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96862a30785b4cdb9d0c2deba5d6da5a" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 36765 } } }
I20260811 06:35:43.641757 32305 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96862a30785b4cdb9d0c2deba5d6da5a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "96862a30785b4cdb9d0c2deba5d6da5a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96862a30785b4cdb9d0c2deba5d6da5a" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 36765 } } }
I20260811 06:35:43.642263 32305 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96862a30785b4cdb9d0c2deba5d6da5a [sys.catalog]: This master's current role is: LEADER
I20260811 06:35:43.642263 32307 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96862a30785b4cdb9d0c2deba5d6da5a [sys.catalog]: This master's current role is: LEADER
I20260811 06:35:43.645499 32313 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:35:43.651896 32313 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:35:43.662217 32313 catalog_manager.cc:1347] Generated new cluster ID: b4e937478bf04963b7ef4242f9b5cdfd
I20260811 06:35:43.662358 32313 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:35:43.695068 32313 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:35:43.695566 32313 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:35:43.704027 32313 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 96862a30785b4cdb9d0c2deba5d6da5a: Generated new TSK 0
I20260811 06:35:43.704869 32313 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:35:43.717546 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 32191
W20260811 06:35:43.718909 32281 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:51211
W20260811 06:35:43.719647 29631 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 32191: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 06:35:43.719825 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 32191
2026-08-11T06:35:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (2337 ms)
[----------] 18 tests from AuthTokenIssuingTest (42945 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429755927508-29631-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:35:43 dist-test-slave-gkw7 krb5kdc[32335](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:35:43 dist-test-slave-gkw7 krb5kdc[32335](info): set up 2 sockets
Aug 11 06:35:43 dist-test-slave-gkw7 krb5kdc[32335](info): commencing operation
krb5kdc: starting...
W20260811 06:35:45.084441 29631 mini_kdc.cc:121] Time spent starting KDC: real 1.311s user 0.002s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429755927508-29631-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429755927508-29631-0/krb5kdc/test-user.keytab.
Aug 11 06:35:45 dist-test-slave-gkw7 krb5kdc[32335](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430145, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:35:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:35:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.239.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.239.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.239.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab.
Entry for principal kudu/127.28.239.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab.
I20260811 06:35:45.452999 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429755927508-29631-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429755927508-29631-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429755927508-29631-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:37695
--webserver_interface=127.28.239.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab
--principal=kudu/127.28.239.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:38769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.239.254:37695,127.28.239.253:45509,127.28.239.252:46563
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:35:45.815613 32356 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 06:35:45.816007 32356 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:35:45.816114 32356 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:35:45.816267 32356 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:35:45.823580 32356 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:35:45.823675 32356 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:35:45.823781 32356 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:35:45.823884 32356 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:35:45.824007 32356 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:35:45.834065 32356 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:38769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429755927508-29631-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429755927508-29631-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.239.254:37695,127.28.239.253:45509,127.28.239.252:46563
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab
--principal=kudu/127.28.239.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:37695
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.239.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429755927508-29631-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:35:45.835451 32356 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:35:45.836856 32356 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:35:45.848963 32365 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:35:45.849141 32366 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:35:45.849141 32368 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:35:45.850636 32356 server_base.cc:1034] running on GCE node
I20260811 06:35:45.851500 32356 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:35:45.852952 32356 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:35:45.854211 32356 hybrid_clock.cc:648] HybridClock initialized: now 1786430145854144 us; error 43 us; skew 500 ppm
I20260811 06:35:45.854574 32356 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:35:45 dist-test-slave-gkw7 krb5kdc[32335](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430145, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:35:45.861869 32356 init.cc:377] Logged in from keytab as kudu/127.28.239.254@KRBTEST.COM (short username kudu)
I20260811 06:35:45.863464 32356 webserver.cc:460] Webserver started at http://127.28.239.254:38011/ using document root <none> and password file <none>
I20260811 06:35:45.864168 32356 fs_manager.cc:359] Metadata directory not provided
I20260811 06:35:45.864262 32356 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:35:45.864542 32356 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:35:45.867594 32356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429755927508-29631-0/minicluster-data/master-0/data/instance:
uuid: "12c9fcd222be4b33bbc9bec088ec17a1"
format_stamp: "Formatted at 2026-08-11 06:35:45 on dist-test-slave-gkw7"
server_key: "033ac60179c797ed45ba58ca06c0114a"
server_key_iv: "e6fa17d1f77ba72c1c258a95cd0aae7c"
server_key_version: "encryptionkey@0"
I20260811 06:35:45.868489 32356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429755927508-29631-0/minicluster-data/master-0/wal/instance:
uuid: "12c9fcd222be4b33bbc9bec088ec17a1"
format_stamp: "Formatted at 2026-08-11 06:35:45 on dist-test-slave-gkw7"
server_key: "033ac60179c797ed45ba58ca06c0114a"
server_key_iv: "e6fa17d1f77ba72c1c258a95cd0aae7c"
server_key_version: "encryptionkey@0"
I20260811 06:35:45.873955 32356 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260811 06:35:45.877635 32381 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:35:45.879041 32356 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 06:35:45.879226 32356 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429755927508-29631-0/minicluster-data/master-0/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429755927508-29631-0/minicluster-data/master-0/wal
uuid: "12c9fcd222be4b33bbc9bec088ec17a1"
format_stamp: "Formatted at 2026-08-11 06:35:45 on dist-test-slave-gkw7"
server_key: "033ac60179c797ed45ba58ca06c0114a"
server_key_iv: "e6fa17d1f77ba72c1c258a95cd0aae7c"
server_key_version: "encryptionkey@0"
I20260811 06:35:45.879473 32356 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429755927508-29631-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429755927508-29631-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429755927508-29631-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:35:45.905756 32356 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:35:45.910292 32356 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:35:45.910600 32356 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:35:45.930869 32356 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.254:37695
I20260811 06:35:45.930891 32463 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.254:37695 every 8 connection(s)
I20260811 06:35:45.932299 32356 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
I20260811 06:35:45.936221 32464 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:35:45.939911 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 32356
I20260811 06:35:45.940171 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429755927508-29631-0/minicluster-data/master-0/wal/instance
I20260811 06:35:45.941004 29631 external_mini_cluster.cc:1428] Setting key 2910ec2b53edbdc76f9072e02cea3b60
I20260811 06:35:45.945161 32464 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.239.254" port: 37695 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 06:35:45 dist-test-slave-gkw7 krb5kdc[32335](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430145, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.254@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:35:45.969213 32466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:45.949493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:37695 (local address 127.0.0.1:51372)
0811 06:35:45.950460 (+ 967us) negotiation.cc:107] Waiting for socket to connect
0811 06:35:45.950510 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 06:35:45.952620 (+ 2110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:35:45.953489 (+ 869us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:35:45.953497 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:35:45.954270 (+ 773us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:35:45.954612 (+ 342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:35:45.954629 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:35:45.956219 (+ 1590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:35:45.956223 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:35:45.957298 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:35:45.957310 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:35:45.958349 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:35:45.958356 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:35:45.958534 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:45.959234 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:35:45.959258 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:35:45.961449 (+ 2191us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:35:45.964779 (+ 3330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:35:45.964792 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:35:45.964804 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:35:45.965210 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:35:45.965678 (+ 468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:35:45.965685 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:35:45.965689 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:35:45.965854 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:35:45.966659 (+ 805us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:35:45.966666 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:35:45.966900 (+ 234us) client_negotiation.cc:769] Sending connection context
0811 06:35:45.967143 (+ 243us) client_negotiation.cc:240] Negotiation successful
0811 06:35:45.967764 (+ 621us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":689,"mutex_wait_us":152,"thread_start_us":379,"threads_started":1}
I20260811 06:35:45.969239 32467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:45.949493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51372 (local address 127.28.239.254:37695)
0811 06:35:45.951312 (+ 1819us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:45.951324 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:45.951380 (+ 56us) server_negotiation.cc:407] Connection header received
0811 06:35:45.953008 (+ 1628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:45.953013 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:45.953254 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:45.953468 (+ 214us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:45.954835 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:45.956041 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:45.957453 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:45.958097 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:45.958318 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:45.961750 (+ 3432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:35:45.961758 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:35:45.961769 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:35:45.961810 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:35:45.964633 (+ 2823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:35:45.965357 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:35:45.965363 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:35:45.965366 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:35:45.965505 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:35:45.965973 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:35:45.965978 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:35:45.965981 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:35:45.966515 (+ 534us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:35:45.966743 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0811 06:35:45.967284 (+ 541us) server_negotiation.cc:299] Negotiation successful
0811 06:35:45.967639 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1586,"thread_start_us":1308,"threads_started":1}
I20260811 06:35:45.972306 32464 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.239.253" port: 45509 } has no permanent_uuid. Determining permanent_uuid...
I20260811 06:35:45.973511 32466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:45.973007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.253:45509 (local address 127.0.0.1:34794)
0811 06:35:45.973209 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0811 06:35:45.973343 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.253:45509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260811 06:35:45.973846 32386 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.28.239.253:45509: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.28.239.253@KRBTEST.COM; defaulting to no policy
I20260811 06:35:45.975507 32466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:45.975109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.253:45509 (local address 127.0.0.1:34810)
0811 06:35:45.975292 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0811 06:35:45.975374 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.253:45509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260811 06:35:45.975721 32464 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.239.253:45509: Network error: Client connection negotiation failed: client connection to 127.28.239.253:45509: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.239.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.239.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.253.keytab.
Entry for principal kudu/127.28.239.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.253.keytab.
I20260811 06:35:46.017390 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429755927508-29631-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429755927508-29631-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429755927508-29631-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429755927508-29631-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.253:45509
--webserver_interface=127.28.239.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.253.keytab
--principal=kudu/127.28.239.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:38769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.239.254:37695,127.28.239.253:45509,127.28.239.252:46563
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 06:35:46.025112 32464 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.239.253" port: 45509 } attempt: 1
I20260811 06:35:46.026423 32466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:46.025903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.253:45509 (local address 127.0.0.1:34812)
0811 06:35:46.026124 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0811 06:35:46.026286 (+ 162us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.253:45509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260811 06:35:46.027473 32466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:46.027100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.253:45509 (local address 127.0.0.1:34822)
0811 06:35:46.027267 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0811 06:35:46.027341 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.253:45509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260811 06:35:46.027650 32464 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.239.253:45509: Network error: Client connection negotiation failed: client connection to 127.28.239.253:45509: connect: Connection refused (error 111)
I20260811 06:35:46.095922 32464 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.239.253" port: 45509 } attempt: 2
I20260811 06:35:46.097286 32466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:46.096771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.253:45509 (local address 127.0.0.1:34832)
0811 06:35:46.097035 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0811 06:35:46.097133 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.253:45509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260811 06:35:46.098450 32466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:46.098034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.253:45509 (local address 127.0.0.1:34838)
0811 06:35:46.098220 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0811 06:35:46.098297 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.253:45509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260811 06:35:46.098655 32464 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.239.253:45509: Network error: Client connection negotiation failed: client connection to 127.28.239.253:45509: connect: Connection refused (error 111)
I20260811 06:35:46.189946 32464 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.239.253" port: 45509 } attempt: 3
I20260811 06:35:46.191160 32466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:46.190691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.253:45509 (local address 127.0.0.1:60932)
0811 06:35:46.190914 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0811 06:35:46.191014 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.253:45509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260811 06:35:46.192252 32466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:46.191842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.253:45509 (local address 127.0.0.1:60934)
0811 06:35:46.192034 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0811 06:35:46.192112 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.253:45509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260811 06:35:46.192449 32464 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.239.253:45509: Network error: Client connection negotiation failed: client connection to 127.28.239.253:45509: connect: Connection refused (error 111)
I20260811 06:35:46.335719 32464 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.239.253" port: 45509 } attempt: 4
I20260811 06:35:46.337013 32466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:46.336437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.253:45509 (local address 127.0.0.1:60936)
0811 06:35:46.336780 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0811 06:35:46.336871 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.253:45509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153}
I20260811 06:35:46.338081 32466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:46.337718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.253:45509 (local address 127.0.0.1:60942)
0811 06:35:46.337868 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0811 06:35:46.337959 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.253:45509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260811 06:35:46.338274 32464 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.239.253:45509: Network error: Client connection negotiation failed: client connection to 127.28.239.253:45509: connect: Connection refused (error 111)
W20260811 06:35:46.380654 32475 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 06:35:46.381065 32475 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:35:46.381196 32475 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:35:46.381332 32475 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:35:46.389190 32475 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:35:46.389284 32475 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:35:46.389341 32475 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:35:46.389384 32475 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:35:46.389447 32475 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:35:46.399813 32475 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:38769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429755927508-29631-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429755927508-29631-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.28.239.254:37695,127.28.239.253:45509,127.28.239.252:46563
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.253.keytab
--principal=kudu/127.28.239.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.253:45509
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429755927508-29631-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.28.239.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429755927508-29631-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:35:46.401196 32475 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:35:46.402685 32475 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:35:46.415004 32491 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:35:46.415103 32484 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:35:46.415164 32485 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:35:46.416115 32475 server_base.cc:1034] running on GCE node
I20260811 06:35:46.416955 32475 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:35:46.418489 32475 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:35:46.419709 32475 hybrid_clock.cc:648] HybridClock initialized: now 1786430146419625 us; error 61 us; skew 500 ppm
I20260811 06:35:46.420048 32475 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:35:46 dist-test-slave-gkw7 krb5kdc[32335](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430146, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:35:46.423444 32475 init.cc:377] Logged in from keytab as kudu/127.28.239.253@KRBTEST.COM (short username kudu)
I20260811 06:35:46.424973 32475 webserver.cc:460] Webserver started at http://127.28.239.253:43275/ using document root <none> and password file <none>
I20260811 06:35:46.425733 32475 fs_manager.cc:359] Metadata directory not provided
I20260811 06:35:46.425827 32475 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:35:46.426117 32475 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:35:46.429108 32475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429755927508-29631-0/minicluster-data/master-1/data/instance:
uuid: "2fa142bfb1c74961bccd4cfea6af64e0"
format_stamp: "Formatted at 2026-08-11 06:35:46 on dist-test-slave-gkw7"
server_key: "9f3f3609de8ec3f01aed1e66678cf940"
server_key_iv: "7599e866917db0f3594b2787d727b6cf"
server_key_version: "encryptionkey@0"
I20260811 06:35:46.429949 32475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429755927508-29631-0/minicluster-data/master-1/wal/instance:
uuid: "2fa142bfb1c74961bccd4cfea6af64e0"
format_stamp: "Formatted at 2026-08-11 06:35:46 on dist-test-slave-gkw7"
server_key: "9f3f3609de8ec3f01aed1e66678cf940"
server_key_iv: "7599e866917db0f3594b2787d727b6cf"
server_key_version: "encryptionkey@0"
I20260811 06:35:46.435305 32475 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 06:35:46.438987 32501 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:35:46.440445 32475 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 06:35:46.440629 32475 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429755927508-29631-0/minicluster-data/master-1/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429755927508-29631-0/minicluster-data/master-1/wal
uuid: "2fa142bfb1c74961bccd4cfea6af64e0"
format_stamp: "Formatted at 2026-08-11 06:35:46 on dist-test-slave-gkw7"
server_key: "9f3f3609de8ec3f01aed1e66678cf940"
server_key_iv: "7599e866917db0f3594b2787d727b6cf"
server_key_version: "encryptionkey@0"
I20260811 06:35:46.440863 32475 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429755927508-29631-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429755927508-29631-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429755927508-29631-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:35:46.456764 32475 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:35:46.461165 32475 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:35:46.461478 32475 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:35:46.480798 32475 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.253:45509
I20260811 06:35:46.480818 32591 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.253:45509 every 8 connection(s)
I20260811 06:35:46.482357 32475 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429755927508-29631-0/minicluster-data/master-1/data/info.pb
I20260811 06:35:46.483771 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 32475
I20260811 06:35:46.484000 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429755927508-29631-0/minicluster-data/master-1/wal/instance
I20260811 06:35:46.484552 29631 external_mini_cluster.cc:1428] Setting key b5151c23f4a4e9da30c7344c4da6d36a
I20260811 06:35:46.487648 32593 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:35:46.498183 32593 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.239.254" port: 37695 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 06:35:46 dist-test-slave-gkw7 krb5kdc[32335](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430146, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.253@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.28.239.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.239.252@KRBTEST.COM" created.
I20260811 06:35:46.524324 32598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:46.501857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58938 (local address 127.28.239.254:37695)
0811 06:35:46.502392 (+ 535us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:46.502397 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:46.503859 (+ 1462us) server_negotiation.cc:407] Connection header received
0811 06:35:46.506281 (+ 2422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:46.506289 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:46.506401 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:46.506557 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:46.508462 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:46.509305 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:46.511230 (+ 1925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:46.511737 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:46.511909 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:46.516580 (+ 4671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:35:46.516586 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:35:46.516590 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:35:46.516646 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:35:46.520130 (+ 3484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:35:46.521116 (+ 986us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:35:46.521122 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:35:46.521125 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:35:46.521369 (+ 244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:35:46.521971 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:35:46.521977 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:35:46.521982 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:35:46.522440 (+ 458us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:35:46.522635 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0811 06:35:46.523566 (+ 931us) server_negotiation.cc:299] Negotiation successful
0811 06:35:46.523829 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":424,"thread_start_us":291,"threads_started":1}
I20260811 06:35:46.525451 32599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:46.502657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:37695 (local address 127.0.0.1:58938)
0811 06:35:46.503652 (+ 995us) negotiation.cc:107] Waiting for socket to connect
0811 06:35:46.503704 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 06:35:46.505937 (+ 2233us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:35:46.506860 (+ 923us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:35:46.506870 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:35:46.507493 (+ 623us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:35:46.508191 (+ 698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:35:46.508213 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:35:46.509518 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:35:46.509522 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:35:46.511048 (+ 1526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:35:46.511063 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:35:46.512025 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:35:46.512035 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:35:46.512335 (+ 300us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:46.513214 (+ 879us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:35:46.513242 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:35:46.516292 (+ 3050us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:35:46.520341 (+ 4049us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:35:46.520352 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:35:46.520372 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:35:46.520929 (+ 557us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:35:46.521525 (+ 596us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:35:46.521559 (+ 34us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:35:46.521563 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:35:46.521813 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:35:46.522609 (+ 796us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:35:46.522619 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:35:46.523044 (+ 425us) client_negotiation.cc:769] Sending connection context
0811 06:35:46.523406 (+ 362us) client_negotiation.cc:240] Negotiation successful
0811 06:35:46.524025 (+ 619us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":684,"thread_start_us":396,"threads_started":1}
I20260811 06:35:46.527717 32593 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.239.253" port: 45509 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 06:35:46 dist-test-slave-gkw7 krb5kdc[32335](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430146, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.253@KRBTEST.COM for kudu/127.28.239.253@KRBTEST.COM
I20260811 06:35:46.545025 32599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:46.528493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.253:45509 (local address 127.0.0.1:60944)
0811 06:35:46.528756 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0811 06:35:46.528782 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 06:35:46.528945 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:35:46.530342 (+ 1397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:35:46.530346 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:35:46.530801 (+ 455us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:35:46.530965 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:35:46.530975 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:35:46.532994 (+ 2019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:35:46.532999 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:35:46.534590 (+ 1591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:35:46.534609 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:35:46.535899 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:35:46.535904 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:35:46.536129 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:46.536819 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:35:46.536837 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:35:46.539121 (+ 2284us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:35:46.542441 (+ 3320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:35:46.542446 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:35:46.542450 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:35:46.542828 (+ 378us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:35:46.543225 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:35:46.543230 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:35:46.543233 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:35:46.543386 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:35:46.544005 (+ 619us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:35:46.544009 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:35:46.544158 (+ 149us) client_negotiation.cc:769] Sending connection context
0811 06:35:46.544301 (+ 143us) client_negotiation.cc:240] Negotiation successful
0811 06:35:46.544598 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"mutex_wait_us":35}
I20260811 06:35:46.545122 32602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:46.528942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60944 (local address 127.28.239.253:45509)
0811 06:35:46.529787 (+ 845us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:46.529797 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:46.529830 (+ 33us) server_negotiation.cc:407] Connection header received
0811 06:35:46.529890 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:46.529900 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:46.530150 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:46.530318 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:46.531153 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:46.532822 (+ 1669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:46.534780 (+ 1958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:46.535724 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:46.535867 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:46.539335 (+ 3468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:35:46.539347 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:35:46.539361 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:35:46.539394 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:35:46.542298 (+ 2904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:35:46.542963 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:35:46.542975 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:35:46.542977 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:35:46.543093 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:35:46.543489 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:35:46.543497 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:35:46.543502 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:35:46.543879 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:35:46.544090 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0811 06:35:46.544454 (+ 364us) server_negotiation.cc:299] Negotiation successful
0811 06:35:46.544750 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":63,"server-negotiator.queue_time_us":644,"thread_start_us":450,"threads_started":1}
I20260811 06:35:46.547714 32593 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.239.252" port: 46563 } has no permanent_uuid. Determining permanent_uuid...
I20260811 06:35:46.549158 32599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:46.548535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.252:46563 (local address 127.0.0.1:58170)
0811 06:35:46.548787 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0811 06:35:46.548933 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.252:46563: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
W20260811 06:35:46.549470 32513 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.28.239.252:46563: connect: Connection refused (error 111)
I20260811 06:35:46.551383 32599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:46.550877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.252:46563 (local address 127.0.0.1:58174)
0811 06:35:46.551113 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0811 06:35:46.551225 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.252:46563: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260811 06:35:46.551636 32593 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.239.252:46563: Network error: Client connection negotiation failed: client connection to 127.28.239.252:46563: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.239.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.252.keytab.
Entry for principal kudu/127.28.239.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.252.keytab.
I20260811 06:35:46.564916 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429755927508-29631-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429755927508-29631-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429755927508-29631-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429755927508-29631-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.252:46563
--webserver_interface=127.28.239.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.252.keytab
--principal=kudu/127.28.239.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:38769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.239.254:37695,127.28.239.253:45509,127.28.239.252:46563
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 06:35:46.601015 32593 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.239.252" port: 46563 } attempt: 1
I20260811 06:35:46.602458 32599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:46.601882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.252:46563 (local address 127.0.0.1:58176)
0811 06:35:46.602114 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0811 06:35:46.602258 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.252:46563: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260811 06:35:46.603514 32599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:46.603140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.252:46563 (local address 127.0.0.1:58184)
0811 06:35:46.603299 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0811 06:35:46.603389 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.252:46563: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260811 06:35:46.603766 32593 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.239.252:46563: Network error: Client connection negotiation failed: client connection to 127.28.239.252:46563: connect: Connection refused (error 111)
I20260811 06:35:46.637603 32464 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.239.253" port: 45509 } attempt: 5
Aug 11 06:35:46 dist-test-slave-gkw7 krb5kdc[32335](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430145, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.254@KRBTEST.COM for kudu/127.28.239.253@KRBTEST.COM
I20260811 06:35:46.651547 32602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:46.638532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60952 (local address 127.28.239.253:45509)
0811 06:35:46.638786 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:46.638792 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:46.638815 (+ 23us) server_negotiation.cc:407] Connection header received
0811 06:35:46.638998 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:46.639010 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:46.639078 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:46.639190 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:46.640016 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:46.640733 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:46.641993 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:46.642438 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:46.642579 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:46.646061 (+ 3482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:35:46.646066 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:35:46.646070 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:35:46.646105 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:35:46.648829 (+ 2724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:35:46.649513 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:35:46.649519 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:35:46.649561 (+ 42us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:35:46.649704 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:35:46.650095 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:35:46.650100 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:35:46.650103 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:35:46.650410 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:35:46.650569 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 06:35:46.650973 (+ 404us) server_negotiation.cc:299] Negotiation successful
0811 06:35:46.651201 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260811 06:35:46.651552 32466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:46.638419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.253:45509 (local address 127.0.0.1:60952)
0811 06:35:46.638666 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0811 06:35:46.638689 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 06:35:46.638833 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:35:46.639267 (+ 434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:35:46.639272 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:35:46.639678 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:35:46.639858 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:35:46.639867 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:35:46.640872 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:35:46.640878 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:35:46.641844 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:35:46.641856 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:35:46.642598 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:35:46.642604 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:35:46.642809 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:46.643462 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:35:46.643479 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:35:46.645861 (+ 2382us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:35:46.648986 (+ 3125us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:35:46.648991 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:35:46.648995 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:35:46.649378 (+ 383us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:35:46.649818 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:35:46.649823 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:35:46.649827 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:35:46.649978 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:35:46.650540 (+ 562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:35:46.650545 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:35:46.650706 (+ 161us) client_negotiation.cc:769] Sending connection context
0811 06:35:46.650856 (+ 150us) client_negotiation.cc:240] Negotiation successful
0811 06:35:46.651199 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":85}
I20260811 06:35:46.652611 32464 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.239.252" port: 46563 } has no permanent_uuid. Determining permanent_uuid...
I20260811 06:35:46.653800 32466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:46.653374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.252:46563 (local address 127.0.0.1:58192)
0811 06:35:46.653570 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0811 06:35:46.653662 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.252:46563: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260811 06:35:46.654769 32466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:46.654447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.252:46563 (local address 127.0.0.1:58206)
0811 06:35:46.654601 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0811 06:35:46.654672 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.252:46563: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260811 06:35:46.654963 32464 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.239.252:46563: Network error: Client connection negotiation failed: client connection to 127.28.239.252:46563: connect: Connection refused (error 111)
I20260811 06:35:46.671998 32593 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.239.252" port: 46563 } attempt: 2
I20260811 06:35:46.673233 32599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:46.672727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.252:46563 (local address 127.0.0.1:58212)
0811 06:35:46.672973 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0811 06:35:46.673067 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.252:46563: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260811 06:35:46.674278 32599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:46.673916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.252:46563 (local address 127.0.0.1:58216)
0811 06:35:46.674059 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0811 06:35:46.674138 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.252:46563: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260811 06:35:46.674497 32593 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.239.252:46563: Network error: Client connection negotiation failed: client connection to 127.28.239.252:46563: connect: Connection refused (error 111)
I20260811 06:35:46.706293 32464 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.239.252" port: 46563 } attempt: 1
I20260811 06:35:46.707490 32466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:46.707003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.252:46563 (local address 127.0.0.1:58224)
0811 06:35:46.707220 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0811 06:35:46.707322 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.252:46563: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260811 06:35:46.708469 32466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:46.708143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.252:46563 (local address 127.0.0.1:58240)
0811 06:35:46.708295 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0811 06:35:46.708369 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.252:46563: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260811 06:35:46.708688 32464 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.239.252:46563: Network error: Client connection negotiation failed: client connection to 127.28.239.252:46563: connect: Connection refused (error 111)
I20260811 06:35:46.765715 32593 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.239.252" port: 46563 } attempt: 3
I20260811 06:35:46.766845 32599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:46.766380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.252:46563 (local address 127.0.0.1:58242)
0811 06:35:46.766584 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0811 06:35:46.766691 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.252:46563: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260811 06:35:46.767925 32599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:46.767494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.252:46563 (local address 127.0.0.1:58252)
0811 06:35:46.767724 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0811 06:35:46.767808 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.252:46563: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130}
W20260811 06:35:46.768129 32593 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.239.252:46563: Network error: Client connection negotiation failed: client connection to 127.28.239.252:46563: connect: Connection refused (error 111)
I20260811 06:35:46.776980 32464 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.239.252" port: 46563 } attempt: 2
I20260811 06:35:46.778126 32466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:46.777662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.252:46563 (local address 127.0.0.1:58256)
0811 06:35:46.777886 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0811 06:35:46.777968 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.252:46563: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260811 06:35:46.779093 32466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:46.778758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.252:46563 (local address 127.0.0.1:58260)
0811 06:35:46.778905 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0811 06:35:46.778978 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.252:46563: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 06:35:46.779327 32464 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.239.252:46563: Network error: Client connection negotiation failed: client connection to 127.28.239.252:46563: connect: Connection refused (error 111)
I20260811 06:35:46.885579 32464 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.239.252" port: 46563 } attempt: 3
I20260811 06:35:46.886710 32466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:46.886232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.252:46563 (local address 127.0.0.1:58264)
0811 06:35:46.886440 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0811 06:35:46.886539 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.252:46563: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260811 06:35:46.887732 32466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:46.887348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.252:46563 (local address 127.0.0.1:58278)
0811 06:35:46.887562 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0811 06:35:46.887633 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.252:46563: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
W20260811 06:35:46.887995 32464 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.239.252:46563: Network error: Client connection negotiation failed: client connection to 127.28.239.252:46563: connect: Connection refused (error 111)
I20260811 06:35:46.911365 32593 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.239.252" port: 46563 } attempt: 4
I20260811 06:35:46.912533 32599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:46.912028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.252:46563 (local address 127.0.0.1:58282)
0811 06:35:46.912283 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0811 06:35:46.912380 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.252:46563: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260811 06:35:46.913579 32599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:46.913222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.252:46563 (local address 127.0.0.1:58298)
0811 06:35:46.913369 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0811 06:35:46.913441 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.252:46563: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 06:35:46.913856 32593 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.239.252:46563: Network error: Client connection negotiation failed: client connection to 127.28.239.252:46563: connect: Connection refused (error 111)
W20260811 06:35:46.925460 32609 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 06:35:46.925879 32609 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:35:46.925987 32609 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:35:46.926111 32609 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:35:46.935624 32609 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:35:46.935736 32609 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:35:46.935823 32609 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:35:46.935900 32609 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:35:46.935977 32609 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:35:46.948354 32609 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:38769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429755927508-29631-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429755927508-29631-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.28.239.254:37695,127.28.239.253:45509,127.28.239.252:46563
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.252.keytab
--principal=kudu/127.28.239.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.252:46563
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429755927508-29631-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.28.239.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429755927508-29631-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:35:46.949712 32609 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:35:46.951043 32609 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:35:46.963364 32617 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:35:46.963338 32618 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:35:46.963507 32621 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:35:46.964643 32609 server_base.cc:1034] running on GCE node
I20260811 06:35:46.965503 32609 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:35:46.967072 32609 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:35:46.968340 32609 hybrid_clock.cc:648] HybridClock initialized: now 1786430146968277 us; error 42 us; skew 500 ppm
I20260811 06:35:46.968699 32609 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:35:46 dist-test-slave-gkw7 krb5kdc[32335](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430146, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:35:46.972023 32609 init.cc:377] Logged in from keytab as kudu/127.28.239.252@KRBTEST.COM (short username kudu)
I20260811 06:35:46.973573 32609 webserver.cc:460] Webserver started at http://127.28.239.252:43395/ using document root <none> and password file <none>
I20260811 06:35:46.974331 32609 fs_manager.cc:359] Metadata directory not provided
I20260811 06:35:46.974475 32609 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:35:46.974833 32609 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:35:46.978049 32609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429755927508-29631-0/minicluster-data/master-2/data/instance:
uuid: "208bc37af3bf4b7d9b155285644f83ac"
format_stamp: "Formatted at 2026-08-11 06:35:46 on dist-test-slave-gkw7"
server_key: "c4ffc0a216d5687ce5635f9ef32f03f6"
server_key_iv: "0fcb38b34ca2d29c98f9d6bff1aa749e"
server_key_version: "encryptionkey@0"
I20260811 06:35:46.978925 32609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429755927508-29631-0/minicluster-data/master-2/wal/instance:
uuid: "208bc37af3bf4b7d9b155285644f83ac"
format_stamp: "Formatted at 2026-08-11 06:35:46 on dist-test-slave-gkw7"
server_key: "c4ffc0a216d5687ce5635f9ef32f03f6"
server_key_iv: "0fcb38b34ca2d29c98f9d6bff1aa749e"
server_key_version: "encryptionkey@0"
I20260811 06:35:46.984724 32609 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 06:35:46.988452 32632 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:35:46.989943 32609 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:35:46.990135 32609 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429755927508-29631-0/minicluster-data/master-2/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429755927508-29631-0/minicluster-data/master-2/wal
uuid: "208bc37af3bf4b7d9b155285644f83ac"
format_stamp: "Formatted at 2026-08-11 06:35:46 on dist-test-slave-gkw7"
server_key: "c4ffc0a216d5687ce5635f9ef32f03f6"
server_key_iv: "0fcb38b34ca2d29c98f9d6bff1aa749e"
server_key_version: "encryptionkey@0"
I20260811 06:35:46.990406 32609 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429755927508-29631-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429755927508-29631-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429755927508-29631-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:35:47.004040 32609 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:35:47.008695 32609 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:35:47.009024 32609 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:35:47.029901 32609 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.252:46563
I20260811 06:35:47.029923 32714 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.252:46563 every 8 connection(s)
I20260811 06:35:47.031463 32609 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429755927508-29631-0/minicluster-data/master-2/data/info.pb
I20260811 06:35:47.035544 32715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:35:47.040899 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 32609
I20260811 06:35:47.041148 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786429755927508-29631-0/minicluster-data/master-2/wal/instance
I20260811 06:35:47.041738 29631 external_mini_cluster.cc:1428] Setting key eed5ea883cff4256cf4975b4d90529dc
I20260811 06:35:47.043577 32715 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.239.254" port: 37695 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 06:35:47 dist-test-slave-gkw7 krb5kdc[32335](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430146, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.252@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
Aug 11 06:35:47 dist-test-slave-gkw7 krb5kdc[32335](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430145, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.239.252@KRBTEST.COM
I20260811 06:35:47.065229 32464 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.239.252" port: 46563 } attempt: 4
I20260811 06:35:47.067510 32722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:47.046875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58942 (local address 127.28.239.254:37695)
0811 06:35:47.047478 (+ 603us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:47.047485 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:47.047911 (+ 426us) server_negotiation.cc:407] Connection header received
0811 06:35:47.048181 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:47.048186 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:47.048277 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:47.048390 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:47.050197 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:47.051342 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:47.053671 (+ 2329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:47.054335 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:47.054477 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:47.059414 (+ 4937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:35:47.059419 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:35:47.059423 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:35:47.059468 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:35:47.063087 (+ 3619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:35:47.064099 (+ 1012us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:35:47.064104 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:35:47.064106 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:35:47.064248 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:35:47.064798 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:35:47.064803 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:35:47.064806 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:35:47.065221 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:35:47.065382 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 06:35:47.066461 (+ 1079us) server_negotiation.cc:299] Negotiation successful
0811 06:35:47.066691 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":462,"thread_start_us":272,"threads_started":1}
I20260811 06:35:47.068475 32721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:47.043838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58308 (local address 127.28.239.252:46563)
0811 06:35:47.044746 (+ 908us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:47.044759 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:47.044842 (+ 83us) server_negotiation.cc:407] Connection header received
0811 06:35:47.047143 (+ 2301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:47.047183 (+ 40us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:47.047768 (+ 585us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:47.048141 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:47.050018 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:47.051780 (+ 1762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:47.053657 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:47.054603 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:47.054764 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:47.059960 (+ 5196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:35:47.059972 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:35:47.059995 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:35:47.060037 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:35:47.063672 (+ 3635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:35:47.064529 (+ 857us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:35:47.064536 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:35:47.064538 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:35:47.064693 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:35:47.065111 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:35:47.065116 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:35:47.065119 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:35:47.065903 (+ 784us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:35:47.066262 (+ 359us) server_negotiation.cc:1035] Waiting for connection context
0811 06:35:47.067187 (+ 925us) server_negotiation.cc:299] Negotiation successful
0811 06:35:47.067445 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":631,"thread_start_us":415,"threads_started":1}
I20260811 06:35:47.068503 32725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:47.046910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:37695 (local address 127.0.0.1:58942)
0811 06:35:47.047752 (+ 842us) negotiation.cc:107] Waiting for socket to connect
0811 06:35:47.047797 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 06:35:47.048015 (+ 218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:35:47.048488 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:35:47.048496 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:35:47.049424 (+ 928us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:35:47.050017 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:35:47.050035 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:35:47.051535 (+ 1500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:35:47.051539 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:35:47.053462 (+ 1923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:35:47.053479 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:35:47.054611 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:35:47.054617 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:35:47.054823 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:47.055735 (+ 912us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:35:47.055767 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:35:47.059176 (+ 3409us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:35:47.063276 (+ 4100us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:35:47.063293 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:35:47.063305 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:35:47.063946 (+ 641us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:35:47.064371 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:35:47.064376 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:35:47.064379 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:35:47.064669 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:35:47.065377 (+ 708us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:35:47.065386 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:35:47.065990 (+ 604us) client_negotiation.cc:769] Sending connection context
0811 06:35:47.066350 (+ 360us) client_negotiation.cc:240] Negotiation successful
0811 06:35:47.066901 (+ 551us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":626,"mutex_wait_us":55,"thread_start_us":459,"threads_started":1}
I20260811 06:35:47.070612 32715 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.239.253" port: 45509 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 06:35:47 dist-test-slave-gkw7 krb5kdc[32335](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430145, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.254@KRBTEST.COM for kudu/127.28.239.252@KRBTEST.COM
Aug 11 06:35:47 dist-test-slave-gkw7 krb5kdc[32335](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430146, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.252@KRBTEST.COM for kudu/127.28.239.253@KRBTEST.COM
I20260811 06:35:47.084528 32466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:47.065928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.252:46563 (local address 127.0.0.1:58318)
0811 06:35:47.066539 (+ 611us) negotiation.cc:107] Waiting for socket to connect
0811 06:35:47.066570 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0811 06:35:47.066671 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:35:47.069310 (+ 2639us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:35:47.069317 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:35:47.069845 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:35:47.070082 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:35:47.070102 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:35:47.071577 (+ 1475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:35:47.071583 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:35:47.072996 (+ 1413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:35:47.073014 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:35:47.074032 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:35:47.074039 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:35:47.074263 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:47.075032 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:35:47.075060 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:35:47.077770 (+ 2710us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:35:47.081314 (+ 3544us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:35:47.081320 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:35:47.081323 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:35:47.081866 (+ 543us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:35:47.082370 (+ 504us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:35:47.082376 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:35:47.082381 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:35:47.082555 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:35:47.083387 (+ 832us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:35:47.083393 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:35:47.083587 (+ 194us) client_negotiation.cc:769] Sending connection context
0811 06:35:47.083750 (+ 163us) client_negotiation.cc:240] Negotiation successful
0811 06:35:47.084073 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483}
I20260811 06:35:47.084759 32721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:47.066222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58318 (local address 127.28.239.252:46563)
0811 06:35:47.068954 (+ 2732us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:47.068959 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:47.068978 (+ 19us) server_negotiation.cc:407] Connection header received
0811 06:35:47.069031 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:47.069039 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:47.069109 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:47.069919 (+ 810us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:47.070230 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:47.071369 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:47.073216 (+ 1847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:47.073834 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:47.073986 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:47.077995 (+ 4009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:35:47.078002 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:35:47.078007 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:35:47.078045 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:35:47.081158 (+ 3113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:35:47.082005 (+ 847us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:35:47.082010 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:35:47.082014 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:35:47.082229 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:35:47.082702 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:35:47.082709 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:35:47.082713 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:35:47.083132 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:35:47.083290 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0811 06:35:47.083893 (+ 603us) server_negotiation.cc:299] Negotiation successful
0811 06:35:47.084350 (+ 457us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":2630,"thread_start_us":3100,"threads_started":1}
Aug 11 06:35:47 dist-test-slave-gkw7 krb5kdc[32335](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430145, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:35:47.086839 32725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:47.071487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.253:45509 (local address 127.0.0.1:60954)
0811 06:35:47.071721 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0811 06:35:47.071744 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 06:35:47.071836 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:35:47.072368 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:35:47.072371 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:35:47.072718 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:35:47.072866 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:35:47.072873 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:35:47.073915 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:35:47.073924 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:35:47.075043 (+ 1119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:35:47.075055 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:35:47.075790 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:35:47.075795 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:35:47.075988 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:47.076575 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:35:47.076601 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:35:47.079419 (+ 2818us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:35:47.083778 (+ 4359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:35:47.083784 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:35:47.083787 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:35:47.084421 (+ 634us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:35:47.084935 (+ 514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:35:47.084940 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:35:47.084942 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:35:47.085159 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:35:47.085881 (+ 722us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:35:47.085884 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:35:47.086012 (+ 128us) client_negotiation.cc:769] Sending connection context
0811 06:35:47.086232 (+ 220us) client_negotiation.cc:240] Negotiation successful
0811 06:35:47.086520 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134}
I20260811 06:35:47.086990 32602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:47.071661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60954 (local address 127.28.239.253:45509)
0811 06:35:47.071871 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:47.071879 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:47.071899 (+ 20us) server_negotiation.cc:407] Connection header received
0811 06:35:47.071964 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:47.071973 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:47.072073 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:47.072179 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:47.073018 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:47.073777 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:47.075203 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:47.075644 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:47.075780 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:47.079637 (+ 3857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:35:47.079646 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:35:47.079652 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:35:47.079701 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:35:47.083573 (+ 3872us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:35:47.084579 (+ 1006us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:35:47.084600 (+ 21us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:35:47.084603 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:35:47.084779 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:35:47.085292 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:35:47.085297 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:35:47.085300 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:35:47.085738 (+ 438us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:35:47.085897 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 06:35:47.086367 (+ 470us) server_negotiation.cc:299] Negotiation successful
0811 06:35:47.086600 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 06:35:47.088176 32715 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.239.252" port: 46563 } has no permanent_uuid. Determining permanent_uuid...
I20260811 06:35:47.092022 32464 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 12c9fcd222be4b33bbc9bec088ec17a1: Bootstrap starting.
I20260811 06:35:47.093693 32722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:47.074045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58958 (local address 127.28.239.254:37695)
0811 06:35:47.074222 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:47.074228 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:47.074247 (+ 19us) server_negotiation.cc:407] Connection header received
0811 06:35:47.074292 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:47.074298 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:47.074394 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:47.074502 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:47.076550 (+ 2048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:47.077714 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:47.080323 (+ 2609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:47.080817 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:47.080970 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:47.086980 (+ 6010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:35:47.086988 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:35:47.086992 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:35:47.087034 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:35:47.090690 (+ 3656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:35:47.091408 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:35:47.091415 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:35:47.091419 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:35:47.091553 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:35:47.091958 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:35:47.091963 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:35:47.091967 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:35:47.092388 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:35:47.092691 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0811 06:35:47.093053 (+ 362us) server_negotiation.cc:299] Negotiation successful
0811 06:35:47.093299 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 06:35:47.097424 32464 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 12c9fcd222be4b33bbc9bec088ec17a1: Neither blocks nor log segments found. Creating new log.
Aug 11 06:35:47 dist-test-slave-gkw7 krb5kdc[32335](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430146, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.252@KRBTEST.COM for kudu/127.28.239.252@KRBTEST.COM
I20260811 06:35:47.098893 32464 log.cc:824] T 00000000000000000000000000000000 P 12c9fcd222be4b33bbc9bec088ec17a1: Log is configured to *not* fsync() on all Append() calls
I20260811 06:35:47.102744 32464 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 12c9fcd222be4b33bbc9bec088ec17a1: No bootstrap required, opened a new log
I20260811 06:35:47.105044 32725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:47.089123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.252:46563 (local address 127.0.0.1:58330)
0811 06:35:47.089566 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0811 06:35:47.089596 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0811 06:35:47.089697 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:35:47.090050 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:35:47.090054 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:35:47.090552 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:35:47.090753 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:35:47.090765 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:35:47.092160 (+ 1395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:35:47.092166 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:35:47.093482 (+ 1316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:35:47.093497 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:35:47.094489 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:35:47.094497 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:35:47.094778 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:47.095812 (+ 1034us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:35:47.095835 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:35:47.098474 (+ 2639us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:35:47.102288 (+ 3814us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:35:47.102293 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:35:47.102297 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:35:47.102808 (+ 511us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:35:47.103218 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:35:47.103223 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:35:47.103226 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:35:47.103373 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:35:47.103977 (+ 604us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:35:47.103981 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:35:47.104146 (+ 165us) client_negotiation.cc:769] Sending connection context
0811 06:35:47.104293 (+ 147us) client_negotiation.cc:240] Negotiation successful
0811 06:35:47.104631 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308}
I20260811 06:35:47.105094 32721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:47.089374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58330 (local address 127.28.239.252:46563)
0811 06:35:47.089613 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:47.089619 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:47.089641 (+ 22us) server_negotiation.cc:407] Connection header received
0811 06:35:47.089819 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:47.089826 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:47.089902 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:47.090005 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:47.090916 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:47.091994 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:47.093654 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:47.094323 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:47.094454 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:47.098666 (+ 4212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:35:47.098672 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:35:47.098677 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:35:47.098715 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:35:47.102150 (+ 3435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:35:47.102967 (+ 817us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:35:47.102976 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:35:47.102978 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:35:47.103097 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:35:47.103490 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:35:47.103495 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:35:47.103498 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:35:47.103848 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:35:47.104002 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 06:35:47.104400 (+ 398us) server_negotiation.cc:299] Negotiation successful
0811 06:35:47.104773 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":162,"server-negotiator.queue_time_us":62}
Aug 11 06:35:47 dist-test-slave-gkw7 krb5kdc[32335](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430145, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.239.253@KRBTEST.COM
I20260811 06:35:47.110037 32715 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 208bc37af3bf4b7d9b155285644f83ac: Bootstrap starting.
I20260811 06:35:47.113512 32715 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 208bc37af3bf4b7d9b155285644f83ac: Neither blocks nor log segments found. Creating new log.
I20260811 06:35:47.114811 32715 log.cc:824] T 00000000000000000000000000000000 P 208bc37af3bf4b7d9b155285644f83ac: Log is configured to *not* fsync() on all Append() calls
I20260811 06:35:47.114920 32464 raft_consensus.cc:348] T 00000000000000000000000000000000 P 12c9fcd222be4b33bbc9bec088ec17a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12c9fcd222be4b33bbc9bec088ec17a1" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 37695 } } peers { permanent_uuid: "2fa142bfb1c74961bccd4cfea6af64e0" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 45509 } } peers { permanent_uuid: "208bc37af3bf4b7d9b155285644f83ac" member_type: VOTER last_known_addr { host: "127.28.239.252" port: 46563 } }
I20260811 06:35:47.115532 32464 raft_consensus.cc:729] T 00000000000000000000000000000000 P 12c9fcd222be4b33bbc9bec088ec17a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12c9fcd222be4b33bbc9bec088ec17a1, State: Initialized, Role: FOLLOWER
I20260811 06:35:47.116185 32464 consensus_queue.cc:260] T 00000000000000000000000000000000 P 12c9fcd222be4b33bbc9bec088ec17a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12c9fcd222be4b33bbc9bec088ec17a1" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 37695 } } peers { permanent_uuid: "2fa142bfb1c74961bccd4cfea6af64e0" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 45509 } } peers { permanent_uuid: "208bc37af3bf4b7d9b155285644f83ac" member_type: VOTER last_known_addr { host: "127.28.239.252" port: 46563 } }
I20260811 06:35:47.116726 32602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:47.095724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60958 (local address 127.28.239.253:45509)
0811 06:35:47.095910 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:47.095916 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:47.095934 (+ 18us) server_negotiation.cc:407] Connection header received
0811 06:35:47.096205 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:47.096214 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:47.096296 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:47.096405 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:47.098123 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:47.099263 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:47.100788 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:47.101508 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:47.101692 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:47.108438 (+ 6746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:35:47.108446 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:35:47.108451 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:35:47.108500 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:35:47.112318 (+ 3818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:35:47.113310 (+ 992us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:35:47.113317 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:35:47.113321 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:35:47.113569 (+ 248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:35:47.114096 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:35:47.114101 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:35:47.114105 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:35:47.114502 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:35:47.114779 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0811 06:35:47.116074 (+ 1295us) server_negotiation.cc:299] Negotiation successful
0811 06:35:47.116348 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 06:35:47.117260 32731 sys_catalog.cc:455] T 00000000000000000000000000000000 P 12c9fcd222be4b33bbc9bec088ec17a1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12c9fcd222be4b33bbc9bec088ec17a1" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 37695 } } peers { permanent_uuid: "2fa142bfb1c74961bccd4cfea6af64e0" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 45509 } } peers { permanent_uuid: "208bc37af3bf4b7d9b155285644f83ac" member_type: VOTER last_known_addr { host: "127.28.239.252" port: 46563 } } }
I20260811 06:35:47.117733 32731 sys_catalog.cc:458] T 00000000000000000000000000000000 P 12c9fcd222be4b33bbc9bec088ec17a1 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:35:47.118572 32715 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 208bc37af3bf4b7d9b155285644f83ac: No bootstrap required, opened a new log
I20260811 06:35:47.119421 32464 sys_catalog.cc:564] T 00000000000000000000000000000000 P 12c9fcd222be4b33bbc9bec088ec17a1 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:35:47.122781 29631 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 06:35:47.127018 32424 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "12c9fcd222be4b33bbc9bec088ec17a1"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58958
I20260811 06:35:47.127701 32424 raft_consensus.cc:482] T 00000000000000000000000000000000 P 12c9fcd222be4b33bbc9bec088ec17a1 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 06:35:47.127924 32424 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 12c9fcd222be4b33bbc9bec088ec17a1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:35:47.129393 32715 raft_consensus.cc:348] T 00000000000000000000000000000000 P 208bc37af3bf4b7d9b155285644f83ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12c9fcd222be4b33bbc9bec088ec17a1" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 37695 } } peers { permanent_uuid: "2fa142bfb1c74961bccd4cfea6af64e0" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 45509 } } peers { permanent_uuid: "208bc37af3bf4b7d9b155285644f83ac" member_type: VOTER last_known_addr { host: "127.28.239.252" port: 46563 } }
I20260811 06:35:47.129901 32715 raft_consensus.cc:729] T 00000000000000000000000000000000 P 208bc37af3bf4b7d9b155285644f83ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 208bc37af3bf4b7d9b155285644f83ac, State: Initialized, Role: FOLLOWER
I20260811 06:35:47.130463 32715 consensus_queue.cc:260] T 00000000000000000000000000000000 P 208bc37af3bf4b7d9b155285644f83ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12c9fcd222be4b33bbc9bec088ec17a1" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 37695 } } peers { permanent_uuid: "2fa142bfb1c74961bccd4cfea6af64e0" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 45509 } } peers { permanent_uuid: "208bc37af3bf4b7d9b155285644f83ac" member_type: VOTER last_known_addr { host: "127.28.239.252" port: 46563 } }
W20260811 06:35:47.131633 32750 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 12c9fcd222be4b33bbc9bec088ec17a1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 06:35:47.131791 32750 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 06:35:47.132061 32424 raft_consensus.cc:504] T 00000000000000000000000000000000 P 12c9fcd222be4b33bbc9bec088ec17a1 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12c9fcd222be4b33bbc9bec088ec17a1" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 37695 } } peers { permanent_uuid: "2fa142bfb1c74961bccd4cfea6af64e0" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 45509 } } peers { permanent_uuid: "208bc37af3bf4b7d9b155285644f83ac" member_type: VOTER last_known_addr { host: "127.28.239.252" port: 46563 } }
I20260811 06:35:47.132826 32753 sys_catalog.cc:455] T 00000000000000000000000000000000 P 208bc37af3bf4b7d9b155285644f83ac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12c9fcd222be4b33bbc9bec088ec17a1" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 37695 } } peers { permanent_uuid: "2fa142bfb1c74961bccd4cfea6af64e0" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 45509 } } peers { permanent_uuid: "208bc37af3bf4b7d9b155285644f83ac" member_type: VOTER last_known_addr { host: "127.28.239.252" port: 46563 } } }
I20260811 06:35:47.133260 32753 sys_catalog.cc:458] T 00000000000000000000000000000000 P 208bc37af3bf4b7d9b155285644f83ac [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:35:47.134281 32715 sys_catalog.cc:564] T 00000000000000000000000000000000 P 208bc37af3bf4b7d9b155285644f83ac [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:35:47.134552 32424 leader_election.cc:288] T 00000000000000000000000000000000 P 12c9fcd222be4b33bbc9bec088ec17a1 [CANDIDATE]: Term 1 election: Requested vote from peers 2fa142bfb1c74961bccd4cfea6af64e0 (127.28.239.253:45509), 208bc37af3bf4b7d9b155285644f83ac (127.28.239.252:46563)
I20260811 06:35:47.137349 32674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "12c9fcd222be4b33bbc9bec088ec17a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "208bc37af3bf4b7d9b155285644f83ac"
I20260811 06:35:47.137956 32674 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 208bc37af3bf4b7d9b155285644f83ac [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:35:47.142168 32674 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 208bc37af3bf4b7d9b155285644f83ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12c9fcd222be4b33bbc9bec088ec17a1 in term 1.
I20260811 06:35:47.143093 32385 leader_election.cc:302] T 00000000000000000000000000000000 P 12c9fcd222be4b33bbc9bec088ec17a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12c9fcd222be4b33bbc9bec088ec17a1, 208bc37af3bf4b7d9b155285644f83ac; no voters:
I20260811 06:35:47.147718 32731 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 12c9fcd222be4b33bbc9bec088ec17a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:35:47.136701 32356 master_runner.cc:186] Error getting master registration for 127.28.239.253:45509: OK, instance_id { permanent_uuid: "2fa142bfb1c74961bccd4cfea6af64e0" instance_seqno: 1786430146475402 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 06:35:47.153019 32356 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 06:35:47.164067 32388 tablet.cc:2369] T 00000000000000000000000000000000 P 12c9fcd222be4b33bbc9bec088ec17a1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:35:47.168618 32731 raft_consensus.cc:686] T 00000000000000000000000000000000 P 12c9fcd222be4b33bbc9bec088ec17a1 [term 1 LEADER]: Becoming Leader. State: Replica: 12c9fcd222be4b33bbc9bec088ec17a1, State: Running, Role: LEADER
I20260811 06:35:47.169608 32731 consensus_queue.cc:237] T 00000000000000000000000000000000 P 12c9fcd222be4b33bbc9bec088ec17a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12c9fcd222be4b33bbc9bec088ec17a1" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 37695 } } peers { permanent_uuid: "2fa142bfb1c74961bccd4cfea6af64e0" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 45509 } } peers { permanent_uuid: "208bc37af3bf4b7d9b155285644f83ac" member_type: VOTER last_known_addr { host: "127.28.239.252" port: 46563 } }
I20260811 06:35:47.189392 32721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:47.148855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58332 (local address 127.28.239.252:46563)
0811 06:35:47.149116 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:47.149125 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:47.163541 (+ 14416us) server_negotiation.cc:407] Connection header received
0811 06:35:47.164704 (+ 1163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:47.164710 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:47.164807 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:47.164891 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:47.168753 (+ 3862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:47.169944 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:47.173187 (+ 3243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:47.173893 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:47.173999 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:47.178603 (+ 4604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:35:47.178611 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:35:47.178616 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:35:47.178667 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:35:47.183222 (+ 4555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:35:47.184818 (+ 1596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:35:47.184826 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:35:47.184830 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:35:47.185102 (+ 272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:35:47.186025 (+ 923us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:35:47.186058 (+ 33us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:35:47.186061 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:35:47.186788 (+ 727us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:35:47.187221 (+ 433us) server_negotiation.cc:1035] Waiting for connection context
0811 06:35:47.188253 (+ 1032us) server_negotiation.cc:299] Negotiation successful
0811 06:35:47.188707 (+ 454us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 06:35:47.206449 32761 sys_catalog.cc:455] T 00000000000000000000000000000000 P 12c9fcd222be4b33bbc9bec088ec17a1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 12c9fcd222be4b33bbc9bec088ec17a1. Latest consensus state: current_term: 1 leader_uuid: "12c9fcd222be4b33bbc9bec088ec17a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12c9fcd222be4b33bbc9bec088ec17a1" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 37695 } } peers { permanent_uuid: "2fa142bfb1c74961bccd4cfea6af64e0" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 45509 } } peers { permanent_uuid: "208bc37af3bf4b7d9b155285644f83ac" member_type: VOTER last_known_addr { host: "127.28.239.252" port: 46563 } } }
I20260811 06:35:47.207051 32761 sys_catalog.cc:458] T 00000000000000000000000000000000 P 12c9fcd222be4b33bbc9bec088ec17a1 [sys.catalog]: This master's current role is: LEADER
I20260811 06:35:47.209637 32602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:47.168488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60966 (local address 127.28.239.253:45509)
0811 06:35:47.168728 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:47.168734 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:47.176499 (+ 7765us) server_negotiation.cc:407] Connection header received
0811 06:35:47.177474 (+ 975us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:47.177482 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:47.177628 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:47.177721 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:47.186756 (+ 9035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:47.187897 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:47.190128 (+ 2231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:47.190815 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:47.190919 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:47.199171 (+ 8252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:35:47.199187 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:35:47.199192 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:35:47.199248 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:35:47.203751 (+ 4503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:35:47.205332 (+ 1581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:35:47.205340 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:35:47.205344 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:35:47.205658 (+ 314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:35:47.206509 (+ 851us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:35:47.206515 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:35:47.206518 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:35:47.207208 (+ 690us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:35:47.207560 (+ 352us) server_negotiation.cc:1035] Waiting for connection context
0811 06:35:47.208574 (+ 1014us) server_negotiation.cc:299] Negotiation successful
0811 06:35:47.208987 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260811 06:35:47.213042 32593 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.239.252" port: 46563 } attempt: 5
I20260811 06:35:47.217819 310 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:35:47.220911 310 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 11 06:35:47 dist-test-slave-gkw7 krb5kdc[32335](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430146, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.253@KRBTEST.COM for kudu/127.28.239.252@KRBTEST.COM
I20260811 06:35:47.223663 32722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:47.158209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58970 (local address 127.28.239.254:37695)
0811 06:35:47.158473 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:47.158482 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:47.165119 (+ 6637us) server_negotiation.cc:407] Connection header received
0811 06:35:47.165221 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:47.165230 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:47.165336 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:47.165468 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:47.173318 (+ 7850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:47.174926 (+ 1608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:47.177723 (+ 2797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:47.178671 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:47.178823 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:47.189122 (+ 10299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:35:47.189132 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:35:47.189139 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:35:47.189200 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:35:47.196112 (+ 6912us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:35:47.217815 (+ 21703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:35:47.217823 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:35:47.217827 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:35:47.218007 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:35:47.221623 (+ 3616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:35:47.221633 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:35:47.221639 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:35:47.222136 (+ 497us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:35:47.222488 (+ 352us) server_negotiation.cc:1035] Waiting for connection context
0811 06:35:47.222988 (+ 500us) server_negotiation.cc:299] Negotiation successful
0811 06:35:47.223268 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260811 06:35:47.239229 32674 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 208bc37af3bf4b7d9b155285644f83ac [term 1 FOLLOWER]: Refusing update from remote peer 12c9fcd222be4b33bbc9bec088ec17a1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:35:47.240500 32761 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 12c9fcd222be4b33bbc9bec088ec17a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "208bc37af3bf4b7d9b155285644f83ac" member_type: VOTER last_known_addr { host: "127.28.239.252" port: 46563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:35:47.241369 32599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:47.213622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.252:46563 (local address 127.0.0.1:58334)
0811 06:35:47.213945 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0811 06:35:47.213966 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 06:35:47.214073 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:35:47.214589 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:35:47.214593 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:35:47.215126 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:35:47.215336 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:35:47.215347 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:35:47.216750 (+ 1403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:35:47.216757 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:35:47.218312 (+ 1555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:35:47.218327 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:35:47.219335 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:35:47.219340 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:35:47.219607 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:47.220358 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:35:47.220385 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:35:47.223315 (+ 2930us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:35:47.237317 (+ 14002us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:35:47.237323 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:35:47.237328 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:35:47.237947 (+ 619us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:35:47.238710 (+ 763us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:35:47.238716 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:35:47.238719 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:35:47.238928 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:35:47.240288 (+ 1360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:35:47.240293 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:35:47.240475 (+ 182us) client_negotiation.cc:769] Sending connection context
0811 06:35:47.240627 (+ 152us) client_negotiation.cc:240] Negotiation successful
0811 06:35:47.240956 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153}
I20260811 06:35:47.244876 32721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:47.213940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58334 (local address 127.28.239.252:46563)
0811 06:35:47.214243 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:47.214252 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:47.214277 (+ 25us) server_negotiation.cc:407] Connection header received
0811 06:35:47.214348 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:47.214358 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:47.214450 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:47.214563 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:47.215486 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:47.216604 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:47.218470 (+ 1866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:47.219138 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:47.219330 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:47.223561 (+ 4231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:35:47.223567 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:35:47.223572 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:35:47.223613 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:35:47.237035 (+ 13422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:35:47.238228 (+ 1193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:35:47.238234 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:35:47.238238 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:35:47.238513 (+ 275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:35:47.239212 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:35:47.239217 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:35:47.239221 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:35:47.240072 (+ 851us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:35:47.242090 (+ 2018us) server_negotiation.cc:1035] Waiting for connection context
0811 06:35:47.242385 (+ 295us) server_negotiation.cc:299] Negotiation successful
0811 06:35:47.242872 (+ 487us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147}
I20260811 06:35:47.252300 32593 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2fa142bfb1c74961bccd4cfea6af64e0: Bootstrap starting.
I20260811 06:35:47.260744 32593 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2fa142bfb1c74961bccd4cfea6af64e0: Neither blocks nor log segments found. Creating new log.
W20260811 06:35:47.261411 32640 tablet.cc:2369] T 00000000000000000000000000000000 P 208bc37af3bf4b7d9b155285644f83ac: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:35:47.264485 32593 log.cc:824] T 00000000000000000000000000000000 P 2fa142bfb1c74961bccd4cfea6af64e0: Log is configured to *not* fsync() on all Append() calls
I20260811 06:35:47.264869 32753 sys_catalog.cc:455] T 00000000000000000000000000000000 P 208bc37af3bf4b7d9b155285644f83ac [sys.catalog]: SysCatalogTable state changed. Reason: New leader 12c9fcd222be4b33bbc9bec088ec17a1. Latest consensus state: current_term: 1 leader_uuid: "12c9fcd222be4b33bbc9bec088ec17a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12c9fcd222be4b33bbc9bec088ec17a1" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 37695 } } peers { permanent_uuid: "2fa142bfb1c74961bccd4cfea6af64e0" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 45509 } } peers { permanent_uuid: "208bc37af3bf4b7d9b155285644f83ac" member_type: VOTER last_known_addr { host: "127.28.239.252" port: 46563 } } }
I20260811 06:35:47.265175 32753 sys_catalog.cc:458] T 00000000000000000000000000000000 P 208bc37af3bf4b7d9b155285644f83ac [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:35:47.269723 313 mvcc.cc:205] Tried to move back new op lower bound from 7317217883075932160 to 7317217882815717376. Current Snapshot: MvccSnapshot[applied={T|T < 7317217883075932160}]
I20260811 06:35:47.269817 32593 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2fa142bfb1c74961bccd4cfea6af64e0: No bootstrap required, opened a new log
I20260811 06:35:47.273510 318 mvcc.cc:205] Tried to move back new op lower bound from 7317217883075932160 to 7317217882815717376. Current Snapshot: MvccSnapshot[applied={T|T < 7317217883075932160}]
I20260811 06:35:47.277303 32761 sys_catalog.cc:455] T 00000000000000000000000000000000 P 12c9fcd222be4b33bbc9bec088ec17a1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "12c9fcd222be4b33bbc9bec088ec17a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12c9fcd222be4b33bbc9bec088ec17a1" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 37695 } } peers { permanent_uuid: "2fa142bfb1c74961bccd4cfea6af64e0" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 45509 } } peers { permanent_uuid: "208bc37af3bf4b7d9b155285644f83ac" member_type: VOTER last_known_addr { host: "127.28.239.252" port: 46563 } } }
I20260811 06:35:47.278787 32761 sys_catalog.cc:458] T 00000000000000000000000000000000 P 12c9fcd222be4b33bbc9bec088ec17a1 [sys.catalog]: This master's current role is: LEADER
I20260811 06:35:47.287160 32753 sys_catalog.cc:455] T 00000000000000000000000000000000 P 208bc37af3bf4b7d9b155285644f83ac [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "12c9fcd222be4b33bbc9bec088ec17a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12c9fcd222be4b33bbc9bec088ec17a1" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 37695 } } peers { permanent_uuid: "2fa142bfb1c74961bccd4cfea6af64e0" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 45509 } } peers { permanent_uuid: "208bc37af3bf4b7d9b155285644f83ac" member_type: VOTER last_known_addr { host: "127.28.239.252" port: 46563 } } }
I20260811 06:35:47.287468 32753 sys_catalog.cc:458] T 00000000000000000000000000000000 P 208bc37af3bf4b7d9b155285644f83ac [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:35:47.292136 310 catalog_manager.cc:1347] Generated new cluster ID: 4b2f9de5d91a4c4582cbf3db49d8f341
I20260811 06:35:47.292271 310 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:35:47.293620 32593 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2fa142bfb1c74961bccd4cfea6af64e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12c9fcd222be4b33bbc9bec088ec17a1" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 37695 } } peers { permanent_uuid: "2fa142bfb1c74961bccd4cfea6af64e0" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 45509 } } peers { permanent_uuid: "208bc37af3bf4b7d9b155285644f83ac" member_type: VOTER last_known_addr { host: "127.28.239.252" port: 46563 } }
I20260811 06:35:47.294513 32593 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2fa142bfb1c74961bccd4cfea6af64e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fa142bfb1c74961bccd4cfea6af64e0, State: Initialized, Role: FOLLOWER
I20260811 06:35:47.296774 32593 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2fa142bfb1c74961bccd4cfea6af64e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12c9fcd222be4b33bbc9bec088ec17a1" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 37695 } } peers { permanent_uuid: "2fa142bfb1c74961bccd4cfea6af64e0" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 45509 } } peers { permanent_uuid: "208bc37af3bf4b7d9b155285644f83ac" member_type: VOTER last_known_addr { host: "127.28.239.252" port: 46563 } }
I20260811 06:35:47.307189 330 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 06:35:47.307854 329 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2fa142bfb1c74961bccd4cfea6af64e0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12c9fcd222be4b33bbc9bec088ec17a1" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 37695 } } peers { permanent_uuid: "2fa142bfb1c74961bccd4cfea6af64e0" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 45509 } } peers { permanent_uuid: "208bc37af3bf4b7d9b155285644f83ac" member_type: VOTER last_known_addr { host: "127.28.239.252" port: 46563 } } }
I20260811 06:35:47.308266 329 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2fa142bfb1c74961bccd4cfea6af64e0 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:35:47.309514 32593 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2fa142bfb1c74961bccd4cfea6af64e0 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:35:47.310292 32548 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2fa142bfb1c74961bccd4cfea6af64e0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:35:47.313705 32550 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "12c9fcd222be4b33bbc9bec088ec17a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "2fa142bfb1c74961bccd4cfea6af64e0"
I20260811 06:35:47.326942 310 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:35:47.329327 32548 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2fa142bfb1c74961bccd4cfea6af64e0 [term 1 FOLLOWER]: Refusing update from remote peer 12c9fcd222be4b33bbc9bec088ec17a1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:35:47.329661 310 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:35:47.333225 32550 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2fa142bfb1c74961bccd4cfea6af64e0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12c9fcd222be4b33bbc9bec088ec17a1 in term 1.
I20260811 06:35:47.335362 32731 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 12c9fcd222be4b33bbc9bec088ec17a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2fa142bfb1c74961bccd4cfea6af64e0" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 45509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:35:47.319203 32609 master_runner.cc:186] Error getting master registration for 127.28.239.253:45509: OK, instance_id { permanent_uuid: "2fa142bfb1c74961bccd4cfea6af64e0" instance_seqno: 1786430146475402 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 06:35:47.354398 310 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 12c9fcd222be4b33bbc9bec088ec17a1: Generated new TSK 0
I20260811 06:35:47.355862 310 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:35:47.358521 32609 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 06:35:47.394913 329 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2fa142bfb1c74961bccd4cfea6af64e0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 12c9fcd222be4b33bbc9bec088ec17a1. Latest consensus state: current_term: 1 leader_uuid: "12c9fcd222be4b33bbc9bec088ec17a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12c9fcd222be4b33bbc9bec088ec17a1" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 37695 } } peers { permanent_uuid: "2fa142bfb1c74961bccd4cfea6af64e0" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 45509 } } peers { permanent_uuid: "208bc37af3bf4b7d9b155285644f83ac" member_type: VOTER last_known_addr { host: "127.28.239.252" port: 46563 } } }
I20260811 06:35:47.395352 329 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2fa142bfb1c74961bccd4cfea6af64e0 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:35:47.394922 336 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2fa142bfb1c74961bccd4cfea6af64e0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "12c9fcd222be4b33bbc9bec088ec17a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12c9fcd222be4b33bbc9bec088ec17a1" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 37695 } } peers { permanent_uuid: "2fa142bfb1c74961bccd4cfea6af64e0" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 45509 } } peers { permanent_uuid: "208bc37af3bf4b7d9b155285644f83ac" member_type: VOTER last_known_addr { host: "127.28.239.252" port: 46563 } } }
I20260811 06:35:47.395588 336 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2fa142bfb1c74961bccd4cfea6af64e0 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:35:47.397272 32761 sys_catalog.cc:455] T 00000000000000000000000000000000 P 12c9fcd222be4b33bbc9bec088ec17a1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "12c9fcd222be4b33bbc9bec088ec17a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12c9fcd222be4b33bbc9bec088ec17a1" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 37695 } } peers { permanent_uuid: "2fa142bfb1c74961bccd4cfea6af64e0" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 45509 } } peers { permanent_uuid: "208bc37af3bf4b7d9b155285644f83ac" member_type: VOTER last_known_addr { host: "127.28.239.252" port: 46563 } } }
I20260811 06:35:47.397603 32761 sys_catalog.cc:458] T 00000000000000000000000000000000 P 12c9fcd222be4b33bbc9bec088ec17a1 [sys.catalog]: This master's current role is: LEADER
I20260811 06:35:47.423149 355 catalog_manager.cc:1259] Loaded cluster ID: 4b2f9de5d91a4c4582cbf3db49d8f341
I20260811 06:35:47.423286 355 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2fa142bfb1c74961bccd4cfea6af64e0: loading cluster ID for follower catalog manager: success
I20260811 06:35:47.427253 355 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2fa142bfb1c74961bccd4cfea6af64e0: acquiring CA information for follower catalog manager: success
I20260811 06:35:47.429829 355 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2fa142bfb1c74961bccd4cfea6af64e0: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260811 06:35:47.474761 32722 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:35:47.468730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58984 (local address 127.28.239.254:37695)
0811 06:35:47.468947 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:47.468952 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:47.472762 (+ 3810us) server_negotiation.cc:407] Connection header received
0811 06:35:47.473087 (+ 325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:47.473094 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:47.473189 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:47.473268 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:47.474492 (+ 1224us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58984: BlockingRecv error: recv got EOF from 127.0.0.1:58984 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260811 06:35:47.475534 32721 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:35:47.468122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58340 (local address 127.28.239.252:46563)
0811 06:35:47.468327 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:47.468334 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:47.472467 (+ 4133us) server_negotiation.cc:407] Connection header received
0811 06:35:47.473998 (+ 1531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:47.474005 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:47.474091 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:47.474159 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:47.475305 (+ 1146us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58340: BlockingRecv error: recv got EOF from 127.0.0.1:58340 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
W20260811 06:35:47.477694 32602 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:35:47.473033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60968 (local address 127.28.239.253:45509)
0811 06:35:47.473261 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:47.473267 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:47.475692 (+ 2425us) server_negotiation.cc:407] Connection header received
0811 06:35:47.476022 (+ 330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:47.476028 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:47.476129 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:47.476201 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:47.477378 (+ 1177us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60968: BlockingRecv error: recv got EOF from 127.0.0.1:60968 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 06:35:47.531615 32602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:47.516447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60976 (local address 127.28.239.253:45509)
0811 06:35:47.516646 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:47.516653 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:47.523563 (+ 6910us) server_negotiation.cc:407] Connection header received
0811 06:35:47.524148 (+ 585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:47.524158 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:47.524380 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:47.524536 (+ 156us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:35:47.525330 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:47.526536 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:47.528605 (+ 2069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:47.529286 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:47.529433 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:47.530084 (+ 651us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:35:47.530819 (+ 735us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:35:47.530970 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 06:35:47.531114 (+ 144us) server_negotiation.cc:299] Negotiation successful
0811 06:35:47.531215 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
W20260811 06:35:47.547938 32721 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:35:47.545609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58356 (local address 127.28.239.252:46563)
0811 06:35:47.545807 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:47.545817 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:47.546235 (+ 418us) server_negotiation.cc:407] Connection header received
0811 06:35:47.546490 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:47.546497 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:47.546655 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:47.546805 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:47.547680 (+ 875us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58356: BlockingRecv error: recv got EOF from 127.0.0.1:58356 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 06:35:47.563472 32602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:47.555954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60978 (local address 127.28.239.253:45509)
0811 06:35:47.556139 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:47.556147 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:47.556810 (+ 663us) server_negotiation.cc:407] Connection header received
0811 06:35:47.557006 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:47.557017 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:47.557142 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:47.557260 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:35:47.557780 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:47.559067 (+ 1287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:47.561153 (+ 2086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:47.561972 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:47.562109 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:47.562607 (+ 498us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:35:47.562784 (+ 177us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:35:47.562884 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 06:35:47.563049 (+ 165us) server_negotiation.cc:299] Negotiation successful
0811 06:35:47.563140 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
W20260811 06:35:47.579964 32721 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:35:47.573160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58360 (local address 127.28.239.252:46563)
0811 06:35:47.573386 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:47.573392 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:47.578272 (+ 4880us) server_negotiation.cc:407] Connection header received
0811 06:35:47.578500 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:47.578507 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:47.578621 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:47.578741 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:47.579721 (+ 980us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58360: BlockingRecv error: recv got EOF from 127.0.0.1:58360 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 06:35:47.595556 32602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:47.587670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60980 (local address 127.28.239.253:45509)
0811 06:35:47.587891 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:47.587897 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:47.588481 (+ 584us) server_negotiation.cc:407] Connection header received
0811 06:35:47.588719 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:47.588725 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:47.588817 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:47.588926 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:35:47.589476 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:47.590780 (+ 1304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:47.592865 (+ 2085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:47.593622 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:47.593803 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:47.594375 (+ 572us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:35:47.594541 (+ 166us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:35:47.594675 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0811 06:35:47.594850 (+ 175us) server_negotiation.cc:299] Negotiation successful
0811 06:35:47.595042 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
W20260811 06:35:47.607991 32721 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:35:47.603083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58372 (local address 127.28.239.252:46563)
0811 06:35:47.603293 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:47.603299 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:47.606265 (+ 2966us) server_negotiation.cc:407] Connection header received
0811 06:35:47.606487 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:47.606494 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:47.606609 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:47.606722 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:47.607729 (+ 1007us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58372: BlockingRecv error: recv got EOF from 127.0.0.1:58372 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 06:35:47.625083 32602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:47.615503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60986 (local address 127.28.239.253:45509)
0811 06:35:47.615769 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:47.615775 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:47.618103 (+ 2328us) server_negotiation.cc:407] Connection header received
0811 06:35:47.618439 (+ 336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:47.618447 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:47.618548 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:47.618672 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:35:47.619314 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:47.620572 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:47.622705 (+ 2133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:47.623498 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:47.623658 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:47.624239 (+ 581us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:35:47.624390 (+ 151us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:35:47.624483 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 06:35:47.624681 (+ 198us) server_negotiation.cc:299] Negotiation successful
0811 06:35:47.624772 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135}
W20260811 06:35:47.638401 32721 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:35:47.636071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58374 (local address 127.28.239.252:46563)
0811 06:35:47.636363 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:47.636371 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:47.636766 (+ 395us) server_negotiation.cc:407] Connection header received
0811 06:35:47.636995 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:47.637001 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:47.637100 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:47.637211 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:47.638162 (+ 951us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58374: BlockingRecv error: recv got EOF from 127.0.0.1:58374 (error 108)
Metrics: {"server-negotiator.queue_time_us":135}
I20260811 06:35:47.658471 32602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:47.650939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60996 (local address 127.28.239.253:45509)
0811 06:35:47.651155 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:47.651162 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:47.651754 (+ 592us) server_negotiation.cc:407] Connection header received
0811 06:35:47.651990 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:47.651996 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:47.652089 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:47.652191 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:35:47.652712 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:47.653925 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:47.655965 (+ 2040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:47.656765 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:47.656903 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:47.657474 (+ 571us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:35:47.657686 (+ 212us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:35:47.657790 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 06:35:47.657995 (+ 205us) server_negotiation.cc:299] Negotiation successful
0811 06:35:47.658092 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
W20260811 06:35:47.668373 32721 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:35:47.666336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58386 (local address 127.28.239.252:46563)
0811 06:35:47.666517 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:47.666525 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:47.666616 (+ 91us) server_negotiation.cc:407] Connection header received
0811 06:35:47.666935 (+ 319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:47.666941 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:47.667040 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:47.667151 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:47.668065 (+ 914us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58386: BlockingRecv error: recv got EOF from 127.0.0.1:58386 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 06:35:47.696117 32602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:47.688801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32780 (local address 127.28.239.253:45509)
0811 06:35:47.689008 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:47.689015 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:47.689613 (+ 598us) server_negotiation.cc:407] Connection header received
0811 06:35:47.689876 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:47.689884 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:47.689973 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:47.690078 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:35:47.690621 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:47.691823 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:47.693853 (+ 2030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:47.694557 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:47.694702 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:47.695262 (+ 560us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:35:47.695437 (+ 175us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:35:47.695533 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 06:35:47.695734 (+ 201us) server_negotiation.cc:299] Negotiation successful
0811 06:35:47.695829 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
W20260811 06:35:47.706173 32721 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:35:47.704019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58396 (local address 127.28.239.252:46563)
0811 06:35:47.704216 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:47.704223 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:47.704347 (+ 124us) server_negotiation.cc:407] Connection header received
0811 06:35:47.704649 (+ 302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:47.704657 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:47.704758 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:47.704868 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:47.705923 (+ 1055us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58396: BlockingRecv error: recv got EOF from 127.0.0.1:58396 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 06:35:47.749053 32602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:47.742721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32784 (local address 127.28.239.253:45509)
0811 06:35:47.742983 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:47.742989 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:47.743326 (+ 337us) server_negotiation.cc:407] Connection header received
0811 06:35:47.743518 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:47.743526 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:47.743639 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:47.743763 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:35:47.744309 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:47.745502 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:47.747002 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:47.747690 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:47.747847 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:47.748246 (+ 399us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:35:47.748386 (+ 140us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:35:47.748484 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 06:35:47.748632 (+ 148us) server_negotiation.cc:299] Negotiation successful
0811 06:35:47.748725 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
W20260811 06:35:47.758414 32721 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:35:47.755287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58398 (local address 127.28.239.252:46563)
0811 06:35:47.755597 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:47.755603 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:47.756520 (+ 917us) server_negotiation.cc:407] Connection header received
0811 06:35:47.756800 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:47.756808 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:47.756919 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:47.757003 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:47.758134 (+ 1131us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58398: BlockingRecv error: recv got EOF from 127.0.0.1:58398 (error 108)
Metrics: {"server-negotiator.queue_time_us":97}
I20260811 06:35:47.833386 32602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:47.826726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32794 (local address 127.28.239.253:45509)
0811 06:35:47.827005 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:47.827013 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:47.827432 (+ 419us) server_negotiation.cc:407] Connection header received
0811 06:35:47.827633 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:47.827637 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:47.827719 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:47.827813 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:35:47.828297 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:47.829162 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:47.831251 (+ 2089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:47.831805 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:47.831916 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:47.832468 (+ 552us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:35:47.832634 (+ 166us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:35:47.832714 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 06:35:47.832919 (+ 205us) server_negotiation.cc:299] Negotiation successful
0811 06:35:47.832992 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
W20260811 06:35:47.841897 32721 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:35:47.839758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58406 (local address 127.28.239.252:46563)
0811 06:35:47.839984 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:47.839991 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:47.840413 (+ 422us) server_negotiation.cc:407] Connection header received
0811 06:35:47.840588 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:47.840594 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:47.840687 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:47.840804 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:47.841640 (+ 836us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58406: BlockingRecv error: recv got EOF from 127.0.0.1:58406 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 06:35:47.979079 32602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:47.973564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32802 (local address 127.28.239.253:45509)
0811 06:35:47.973820 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:47.973828 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:47.974124 (+ 296us) server_negotiation.cc:407] Connection header received
0811 06:35:47.974384 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:47.974388 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:47.974462 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:47.974574 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:35:47.975009 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:47.975822 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:47.977209 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:47.977755 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:47.977901 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:47.978321 (+ 420us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:35:47.978435 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:35:47.978537 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 06:35:47.978689 (+ 152us) server_negotiation.cc:299] Negotiation successful
0811 06:35:47.978776 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
W20260811 06:35:47.987217 32721 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:35:47.985229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58410 (local address 127.28.239.252:46563)
0811 06:35:47.985441 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:47.985445 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:47.985919 (+ 474us) server_negotiation.cc:407] Connection header received
0811 06:35:47.986121 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:47.986125 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:47.986204 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:47.986311 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:47.987030 (+ 719us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58410: BlockingRecv error: recv got EOF from 127.0.0.1:58410 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 06:35:48.252919 32602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:48.247365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32806 (local address 127.28.239.253:45509)
0811 06:35:48.247669 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:48.247677 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:48.248079 (+ 402us) server_negotiation.cc:407] Connection header received
0811 06:35:48.248463 (+ 384us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:48.248469 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:48.248589 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:48.248692 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:35:48.249132 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:48.249897 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:48.251286 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:48.251727 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:48.251835 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:48.252188 (+ 353us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:35:48.252297 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:35:48.252367 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 06:35:48.252509 (+ 142us) server_negotiation.cc:299] Negotiation successful
0811 06:35:48.252577 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
W20260811 06:35:48.262934 32721 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:35:48.260592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58422 (local address 127.28.239.252:46563)
0811 06:35:48.260869 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:48.260877 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:48.261189 (+ 312us) server_negotiation.cc:407] Connection header received
0811 06:35:48.261455 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:48.261462 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:48.261666 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:48.261825 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:48.262732 (+ 907us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58422: BlockingRecv error: recv got EOF from 127.0.0.1:58422 (error 108)
Metrics: {"server-negotiator.queue_time_us":96}
I20260811 06:35:48.312063 330 catalog_manager.cc:1259] Loaded cluster ID: 4b2f9de5d91a4c4582cbf3db49d8f341
I20260811 06:35:48.312225 330 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 208bc37af3bf4b7d9b155285644f83ac: loading cluster ID for follower catalog manager: success
I20260811 06:35:48.315030 330 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 208bc37af3bf4b7d9b155285644f83ac: acquiring CA information for follower catalog manager: success
I20260811 06:35:48.317013 330 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 208bc37af3bf4b7d9b155285644f83ac: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 06:35:48.788594 541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:48.783111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32816 (local address 127.28.239.253:45509)
0811 06:35:48.783667 (+ 556us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:48.783673 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:48.783860 (+ 187us) server_negotiation.cc:407] Connection header received
0811 06:35:48.784043 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:48.784046 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:48.784116 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:48.784246 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:35:48.784645 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:48.785387 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:48.786935 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:48.787351 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:48.787454 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:48.787858 (+ 404us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:35:48.787961 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:35:48.788031 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 06:35:48.788168 (+ 137us) server_negotiation.cc:299] Negotiation successful
0811 06:35:48.788279 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":289,"threads_started":1}
I20260811 06:35:48.800659 549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:48.794763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58426 (local address 127.28.239.252:46563)
0811 06:35:48.795299 (+ 536us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:48.795303 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:48.795321 (+ 18us) server_negotiation.cc:407] Connection header received
0811 06:35:48.795487 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:48.795491 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:48.795666 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:48.795778 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:35:48.796151 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:48.796887 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:48.798313 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:48.798779 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:48.798881 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:48.799445 (+ 564us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:35:48.800047 (+ 602us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:35:48.800191 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 06:35:48.800345 (+ 154us) server_negotiation.cc:299] Negotiation successful
0811 06:35:48.800419 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":424,"thread_start_us":334,"threads_started":1}
I20260811 06:35:48.816401 549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:48.807598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58436 (local address 127.28.239.252:46563)
0811 06:35:48.807853 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:48.807861 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:48.809206 (+ 1345us) server_negotiation.cc:407] Connection header received
0811 06:35:48.809389 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:48.809395 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:48.809511 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:48.809872 (+ 361us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:35:48.811043 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:48.812126 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:48.814067 (+ 1941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:48.814532 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:48.814688 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:48.815483 (+ 795us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:35:48.815660 (+ 177us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:35:48.815756 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 06:35:48.815941 (+ 185us) server_negotiation.cc:299] Negotiation successful
0811 06:35:48.816028 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260811 06:35:48.817251 558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:48.807520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58996 (local address 127.28.239.254:37695)
0811 06:35:48.808005 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:48.808010 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:48.808500 (+ 490us) server_negotiation.cc:407] Connection header received
0811 06:35:48.808714 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:48.808723 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:48.808991 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:48.809581 (+ 590us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:35:48.810296 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:48.811411 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:48.813796 (+ 2385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:48.814496 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:48.814691 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:48.815485 (+ 794us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:35:48.816058 (+ 573us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:35:48.816184 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 06:35:48.816824 (+ 640us) server_negotiation.cc:299] Negotiation successful
0811 06:35:48.816931 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":256,"threads_started":1}
I20260811 06:35:48.818598 541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:48.809340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32822 (local address 127.28.239.253:45509)
0811 06:35:48.809758 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:48.809766 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:48.810584 (+ 818us) server_negotiation.cc:407] Connection header received
0811 06:35:48.810813 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:48.810818 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:48.810936 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:48.811058 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:35:48.813144 (+ 2086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:48.814406 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:48.816489 (+ 2083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:48.817129 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:48.817274 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:48.817785 (+ 511us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:35:48.817922 (+ 137us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:35:48.818030 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 06:35:48.818216 (+ 186us) server_negotiation.cc:299] Negotiation successful
0811 06:35:48.818298 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269}
I20260811 06:35:48.824005 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 32356
I20260811 06:35:48.825191 32456 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:35:48.942574 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 32356
I20260811 06:35:48.970753 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 32475
I20260811 06:35:48.971858 32585 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:35:49.083678 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 32475
I20260811 06:35:49.114692 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 32609
I20260811 06:35:49.116067 32710 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:35:49.227986 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 32609
2026-08-11T06:35:49Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (5508 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429755927508-29631-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:35:49 dist-test-slave-gkw7 krb5kdc[571](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:35:49 dist-test-slave-gkw7 krb5kdc[571](info): set up 2 sockets
Aug 11 06:35:49 dist-test-slave-gkw7 krb5kdc[571](info): commencing operation
krb5kdc: starting...
W20260811 06:35:50.682307 29631 mini_kdc.cc:121] Time spent starting KDC: real 1.400s user 0.008s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429755927508-29631-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429755927508-29631-0/krb5kdc/test-user.keytab.
Aug 11 06:35:50 dist-test-slave-gkw7 krb5kdc[571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430150, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:35:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:35:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.239.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.239.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.239.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab.
Entry for principal kudu/127.28.239.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab.
I20260811 06:35:51.053920 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429755927508-29631-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429755927508-29631-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429755927508-29631-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:37761
--webserver_interface=127.28.239.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab
--principal=kudu/127.28.239.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:34597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.239.254:37761,127.28.239.253:37315,127.28.239.252:34949
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:35:51.412506 588 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 06:35:51.412899 588 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:35:51.413014 588 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:35:51.413146 588 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:35:51.420575 588 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:35:51.420686 588 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:35:51.420791 588 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:35:51.420876 588 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:35:51.420990 588 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:35:51.431355 588 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:34597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429755927508-29631-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429755927508-29631-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.239.254:37761,127.28.239.253:37315,127.28.239.252:34949
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.254.keytab
--principal=kudu/127.28.239.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.254:37761
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.239.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429755927508-29631-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:35:51.432792 588 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:35:51.434255 588 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:35:51.446959 595 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:35:51.447044 602 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:35:51.447080 599 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:35:51.447999 588 server_base.cc:1034] running on GCE node
I20260811 06:35:51.448963 588 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:35:51.450649 588 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:35:51.451918 588 hybrid_clock.cc:648] HybridClock initialized: now 1786430151451830 us; error 61 us; skew 500 ppm
I20260811 06:35:51.452288 588 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:35:51 dist-test-slave-gkw7 krb5kdc[571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430151, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:35:51.459437 588 init.cc:377] Logged in from keytab as kudu/127.28.239.254@KRBTEST.COM (short username kudu)
I20260811 06:35:51.461097 588 webserver.cc:460] Webserver started at http://127.28.239.254:35375/ using document root <none> and password file <none>
I20260811 06:35:51.461990 588 fs_manager.cc:359] Metadata directory not provided
I20260811 06:35:51.462100 588 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:35:51.462411 588 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:35:51.465844 588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429755927508-29631-0/minicluster-data/master-0/data/instance:
uuid: "90e689ca66554b29aea235643641e4ea"
format_stamp: "Formatted at 2026-08-11 06:35:51 on dist-test-slave-gkw7"
server_key: "1f79b7fbb22e0dfa16f28cf11256e780"
server_key_iv: "26eaf78d0e0601604cc5ced9a1658c10"
server_key_version: "encryptionkey@0"
I20260811 06:35:51.466697 588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429755927508-29631-0/minicluster-data/master-0/wal/instance:
uuid: "90e689ca66554b29aea235643641e4ea"
format_stamp: "Formatted at 2026-08-11 06:35:51 on dist-test-slave-gkw7"
server_key: "1f79b7fbb22e0dfa16f28cf11256e780"
server_key_iv: "26eaf78d0e0601604cc5ced9a1658c10"
server_key_version: "encryptionkey@0"
I20260811 06:35:51.472759 588 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260811 06:35:51.476790 616 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:35:51.478367 588 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:35:51.478518 588 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429755927508-29631-0/minicluster-data/master-0/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429755927508-29631-0/minicluster-data/master-0/wal
uuid: "90e689ca66554b29aea235643641e4ea"
format_stamp: "Formatted at 2026-08-11 06:35:51 on dist-test-slave-gkw7"
server_key: "1f79b7fbb22e0dfa16f28cf11256e780"
server_key_iv: "26eaf78d0e0601604cc5ced9a1658c10"
server_key_version: "encryptionkey@0"
I20260811 06:35:51.478715 588 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429755927508-29631-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429755927508-29631-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429755927508-29631-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:35:51.502750 588 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:35:51.507373 588 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:35:51.507617 588 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:35:51.527992 588 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.254:37761
I20260811 06:35:51.528023 688 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.254:37761 every 8 connection(s)
I20260811 06:35:51.529570 588 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429755927508-29631-0/minicluster-data/master-0/data/info.pb
I20260811 06:35:51.531628 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 588
I20260811 06:35:51.531862 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429755927508-29631-0/minicluster-data/master-0/wal/instance
I20260811 06:35:51.532347 29631 external_mini_cluster.cc:1428] Setting key 35539dd1980427d03cd8a6db387ccdaa
I20260811 06:35:51.533982 689 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:35:51.540983 689 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.239.254" port: 37761 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 06:35:51 dist-test-slave-gkw7 krb5kdc[571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430151, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.254@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:35:51.568007 692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:51.544548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52354 (local address 127.28.239.254:37761)
0811 06:35:51.545458 (+ 910us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:51.545479 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:51.546013 (+ 534us) server_negotiation.cc:407] Connection header received
0811 06:35:51.548086 (+ 2073us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:51.548092 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:51.548354 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:51.548584 (+ 230us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:51.550318 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:51.551705 (+ 1387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:51.554102 (+ 2397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:51.554718 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:51.554897 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:51.559711 (+ 4814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:35:51.559720 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:35:51.559734 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:35:51.559774 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:35:51.562938 (+ 3164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:35:51.563950 (+ 1012us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:35:51.563957 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:35:51.563960 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:35:51.564156 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:35:51.564674 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:35:51.564679 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:35:51.564681 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:35:51.565419 (+ 738us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:35:51.565672 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0811 06:35:51.566432 (+ 760us) server_negotiation.cc:299] Negotiation successful
0811 06:35:51.566835 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":52,"server-negotiator.queue_time_us":648,"thread_start_us":409,"threads_started":1}
I20260811 06:35:51.568037 693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:51.544562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:37761 (local address 127.0.0.1:52354)
0811 06:35:51.545837 (+ 1275us) negotiation.cc:107] Waiting for socket to connect
0811 06:35:51.545882 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 06:35:51.547698 (+ 1816us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:35:51.548592 (+ 894us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:35:51.548599 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:35:51.549454 (+ 855us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:35:51.550009 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:35:51.550037 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:35:51.551893 (+ 1856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:35:51.551902 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:35:51.553914 (+ 2012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:35:51.553937 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:35:51.555021 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:35:51.555035 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:35:51.555278 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:51.556047 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:35:51.556081 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:35:51.559426 (+ 3345us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:35:51.563101 (+ 3675us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:35:51.563116 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:35:51.563132 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:35:51.563798 (+ 666us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:35:51.564284 (+ 486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:35:51.564292 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:35:51.564298 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:35:51.564515 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:35:51.565628 (+ 1113us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:35:51.565640 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:35:51.565963 (+ 323us) client_negotiation.cc:769] Sending connection context
0811 06:35:51.566287 (+ 324us) client_negotiation.cc:240] Negotiation successful
0811 06:35:51.567042 (+ 755us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1052,"mutex_wait_us":119,"spinlock_wait_cycles":17792,"thread_start_us":778,"threads_started":1}
WARNING: no policy specified for kudu/127.28.239.253@KRBTEST.COM; defaulting to no policy
I20260811 06:35:51.571334 689 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.239.253" port: 37315 } has no permanent_uuid. Determining permanent_uuid...
I20260811 06:35:51.572625 693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:51.572145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.253:37315 (local address 127.0.0.1:53514)
0811 06:35:51.572308 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0811 06:35:51.572416 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.253:37315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.239.253@KRBTEST.COM" created.
W20260811 06:35:51.572875 618 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.28.239.253:37315: connect: Connection refused (error 111)
I20260811 06:35:51.574724 693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:51.574283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.253:37315 (local address 127.0.0.1:53516)
0811 06:35:51.574505 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0811 06:35:51.574595 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.253:37315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
W20260811 06:35:51.575066 689 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.239.253:37315: Network error: Client connection negotiation failed: client connection to 127.28.239.253:37315: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.239.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.253.keytab.
Entry for principal kudu/127.28.239.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.253.keytab.
I20260811 06:35:51.613685 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429755927508-29631-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429755927508-29631-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429755927508-29631-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429755927508-29631-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.253:37315
--webserver_interface=127.28.239.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.253.keytab
--principal=kudu/127.28.239.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:34597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.239.254:37761,127.28.239.253:37315,127.28.239.252:34949
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 06:35:51.624574 689 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.239.253" port: 37315 } attempt: 1
I20260811 06:35:51.625972 693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:51.625423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.253:37315 (local address 127.0.0.1:53524)
0811 06:35:51.625705 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0811 06:35:51.625839 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.253:37315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260811 06:35:51.627059 693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:51.626711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.253:37315 (local address 127.0.0.1:53526)
0811 06:35:51.626859 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0811 06:35:51.626928 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.253:37315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260811 06:35:51.627357 689 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.239.253:37315: Network error: Client connection negotiation failed: client connection to 127.28.239.253:37315: connect: Connection refused (error 111)
I20260811 06:35:51.695648 689 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.239.253" port: 37315 } attempt: 2
I20260811 06:35:51.697016 693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:51.696459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.253:37315 (local address 127.0.0.1:53532)
0811 06:35:51.696758 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0811 06:35:51.696877 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.253:37315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
I20260811 06:35:51.698359 693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:51.697944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.253:37315 (local address 127.0.0.1:53540)
0811 06:35:51.698124 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0811 06:35:51.698201 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.253:37315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260811 06:35:51.698614 689 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.239.253:37315: Network error: Client connection negotiation failed: client connection to 127.28.239.253:37315: connect: Connection refused (error 111)
I20260811 06:35:51.789891 689 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.239.253" port: 37315 } attempt: 3
I20260811 06:35:51.791283 693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:51.790704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.253:37315 (local address 127.0.0.1:53546)
0811 06:35:51.790922 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0811 06:35:51.791051 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.253:37315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260811 06:35:51.792448 693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:51.792052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.253:37315 (local address 127.0.0.1:53548)
0811 06:35:51.792219 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0811 06:35:51.792305 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.253:37315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260811 06:35:51.792667 689 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.239.253:37315: Network error: Client connection negotiation failed: client connection to 127.28.239.253:37315: connect: Connection refused (error 111)
I20260811 06:35:51.935947 689 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.239.253" port: 37315 } attempt: 4
I20260811 06:35:51.937461 693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:51.936836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.253:37315 (local address 127.0.0.1:53560)
0811 06:35:51.937208 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0811 06:35:51.937305 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.253:37315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183}
I20260811 06:35:51.938858 693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:51.938282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.253:37315 (local address 127.0.0.1:53566)
0811 06:35:51.938547 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0811 06:35:51.938663 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.253:37315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151}
W20260811 06:35:51.939134 689 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.239.253:37315: Network error: Client connection negotiation failed: client connection to 127.28.239.253:37315: connect: Connection refused (error 111)
W20260811 06:35:51.998371 702 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 06:35:51.998741 702 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:35:51.998819 702 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:35:51.998903 702 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:35:52.006395 702 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:35:52.006502 702 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:35:52.006562 702 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:35:52.006609 702 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:35:52.006683 702 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:35:52.017055 702 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:34597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429755927508-29631-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429755927508-29631-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.28.239.254:37761,127.28.239.253:37315,127.28.239.252:34949
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.253.keytab
--principal=kudu/127.28.239.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.253:37315
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429755927508-29631-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.28.239.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429755927508-29631-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:35:52.018512 702 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:35:52.019847 702 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:35:52.032593 713 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:35:52.032668 716 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:35:52.032586 714 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:35:52.034308 702 server_base.cc:1034] running on GCE node
I20260811 06:35:52.035113 702 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:35:52.036597 702 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:35:52.037829 702 hybrid_clock.cc:648] HybridClock initialized: now 1786430152037760 us; error 34 us; skew 500 ppm
I20260811 06:35:52.038205 702 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:35:52 dist-test-slave-gkw7 krb5kdc[571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430152, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:35:52.041643 702 init.cc:377] Logged in from keytab as kudu/127.28.239.253@KRBTEST.COM (short username kudu)
I20260811 06:35:52.043329 702 webserver.cc:460] Webserver started at http://127.28.239.253:44459/ using document root <none> and password file <none>
I20260811 06:35:52.044083 702 fs_manager.cc:359] Metadata directory not provided
I20260811 06:35:52.044229 702 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:35:52.044585 702 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:35:52.047819 702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429755927508-29631-0/minicluster-data/master-1/data/instance:
uuid: "dac31c807aa04f0ab3869b1f943ecb0b"
format_stamp: "Formatted at 2026-08-11 06:35:52 on dist-test-slave-gkw7"
server_key: "23734a5f319e0f822f8c6ea8c17e5ab7"
server_key_iv: "021681a3da7f523c2d0578268f98e3f1"
server_key_version: "encryptionkey@0"
I20260811 06:35:52.048660 702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429755927508-29631-0/minicluster-data/master-1/wal/instance:
uuid: "dac31c807aa04f0ab3869b1f943ecb0b"
format_stamp: "Formatted at 2026-08-11 06:35:52 on dist-test-slave-gkw7"
server_key: "23734a5f319e0f822f8c6ea8c17e5ab7"
server_key_iv: "021681a3da7f523c2d0578268f98e3f1"
server_key_version: "encryptionkey@0"
I20260811 06:35:52.054273 702 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 06:35:52.058094 727 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:35:52.059536 702 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 06:35:52.059743 702 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429755927508-29631-0/minicluster-data/master-1/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429755927508-29631-0/minicluster-data/master-1/wal
uuid: "dac31c807aa04f0ab3869b1f943ecb0b"
format_stamp: "Formatted at 2026-08-11 06:35:52 on dist-test-slave-gkw7"
server_key: "23734a5f319e0f822f8c6ea8c17e5ab7"
server_key_iv: "021681a3da7f523c2d0578268f98e3f1"
server_key_version: "encryptionkey@0"
I20260811 06:35:52.059978 702 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429755927508-29631-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429755927508-29631-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429755927508-29631-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:35:52.096252 702 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:35:52.101187 702 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:35:52.101498 702 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:35:52.121816 702 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.253:37315
I20260811 06:35:52.121825 802 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.253:37315 every 8 connection(s)
I20260811 06:35:52.123292 702 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429755927508-29631-0/minicluster-data/master-1/data/info.pb
I20260811 06:35:52.127221 803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:35:52.132513 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 702
I20260811 06:35:52.132747 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429755927508-29631-0/minicluster-data/master-1/wal/instance
I20260811 06:35:52.133219 29631 external_mini_cluster.cc:1428] Setting key 095960751bb425a805a64482eb54709d
I20260811 06:35:52.134402 803 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.239.254" port: 37761 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 06:35:52 dist-test-slave-gkw7 krb5kdc[571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430152, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.253@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:35:52.157033 806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:52.137409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52360 (local address 127.28.239.254:37761)
0811 06:35:52.138009 (+ 600us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:52.138014 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:52.139145 (+ 1131us) server_negotiation.cc:407] Connection header received
0811 06:35:52.141336 (+ 2191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:52.141343 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:52.141449 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:52.141620 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:52.143415 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:52.144199 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:52.145980 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:52.146426 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:52.146600 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:52.150608 (+ 4008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:35:52.150614 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:35:52.150617 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:35:52.150673 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:35:52.153641 (+ 2968us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:35:52.154484 (+ 843us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:35:52.154492 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:35:52.154501 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:35:52.154630 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:35:52.155126 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:35:52.155130 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:35:52.155132 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:35:52.155432 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:35:52.155604 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0811 06:35:52.156380 (+ 776us) server_negotiation.cc:299] Negotiation successful
0811 06:35:52.156625 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":484,"thread_start_us":351,"threads_started":1}
I20260811 06:35:52.158257 807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:52.138089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:37761 (local address 127.0.0.1:52360)
0811 06:35:52.138945 (+ 856us) negotiation.cc:107] Waiting for socket to connect
0811 06:35:52.138996 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 06:35:52.140991 (+ 1995us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:35:52.141908 (+ 917us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:35:52.141917 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:35:52.142512 (+ 595us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:35:52.143160 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:35:52.143180 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:35:52.144399 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:35:52.144403 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:35:52.145795 (+ 1392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:35:52.145810 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:35:52.146711 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:35:52.146720 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:35:52.146988 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:52.147818 (+ 830us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:35:52.147844 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:35:52.150324 (+ 2480us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:35:52.153803 (+ 3479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:35:52.153813 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:35:52.153833 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:35:52.154339 (+ 506us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:35:52.154767 (+ 428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:35:52.154772 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:35:52.154777 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:35:52.154985 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:35:52.155583 (+ 598us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:35:52.155592 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:35:52.155971 (+ 379us) client_negotiation.cc:769] Sending connection context
0811 06:35:52.156268 (+ 297us) client_negotiation.cc:240] Negotiation successful
0811 06:35:52.156851 (+ 583us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":594,"thread_start_us":360,"threads_started":1}
I20260811 06:35:52.160243 803 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.239.253" port: 37315 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.28.239.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.239.252@KRBTEST.COM" created.
Aug 11 06:35:52 dist-test-slave-gkw7 krb5kdc[571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430152, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.253@KRBTEST.COM for kudu/127.28.239.253@KRBTEST.COM
I20260811 06:35:52.176651 807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:52.160978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.253:37315 (local address 127.0.0.1:53572)
0811 06:35:52.161172 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0811 06:35:52.161187 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 06:35:52.161300 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:35:52.162489 (+ 1189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:35:52.162494 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:35:52.162951 (+ 457us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:35:52.163085 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:35:52.163094 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:35:52.164693 (+ 1599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:35:52.164697 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:35:52.165611 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:35:52.165624 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:35:52.166451 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:35:52.166456 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:35:52.166703 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:52.167362 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:35:52.167379 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:35:52.170815 (+ 3436us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:35:52.174082 (+ 3267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:35:52.174087 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:35:52.174091 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:35:52.174459 (+ 368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:35:52.174861 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:35:52.174865 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:35:52.174869 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:35:52.174998 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:35:52.175606 (+ 608us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:35:52.175616 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:35:52.175767 (+ 151us) client_negotiation.cc:769] Sending connection context
0811 06:35:52.175922 (+ 155us) client_negotiation.cc:240] Negotiation successful
0811 06:35:52.176228 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62,"mutex_wait_us":25}
I20260811 06:35:52.176708 810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:52.161340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53572 (local address 127.28.239.253:37315)
0811 06:35:52.162075 (+ 735us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:52.162081 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:52.162106 (+ 25us) server_negotiation.cc:407] Connection header received
0811 06:35:52.162159 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:52.162165 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:52.162339 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:52.162505 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:52.163267 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:52.164515 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:52.165754 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:52.166303 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:52.166443 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:52.171037 (+ 4594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:35:52.171049 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:35:52.171064 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:35:52.171093 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:35:52.173939 (+ 2846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:35:52.174594 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:35:52.174605 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:35:52.174609 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:35:52.174741 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:35:52.175108 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:35:52.175112 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:35:52.175116 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:35:52.175481 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:35:52.175712 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0811 06:35:52.176080 (+ 368us) server_negotiation.cc:299] Negotiation successful
0811 06:35:52.176338 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":46,"server-negotiator.queue_time_us":581,"thread_start_us":431,"threads_started":1}
I20260811 06:35:52.179819 803 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.239.252" port: 34949 } has no permanent_uuid. Determining permanent_uuid...
I20260811 06:35:52.181272 807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:52.180655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.252:34949 (local address 127.0.0.1:45824)
0811 06:35:52.180992 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0811 06:35:52.181090 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.252:34949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173}
W20260811 06:35:52.181694 733 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.28.239.252:34949: connect: Connection refused (error 111)
I20260811 06:35:52.183432 807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:52.183072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.252:34949 (local address 127.0.0.1:45838)
0811 06:35:52.183256 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0811 06:35:52.183328 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.252:34949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 06:35:52.183670 803 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.239.252:34949: Network error: Client connection negotiation failed: client connection to 127.28.239.252:34949: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.239.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.252.keytab.
Entry for principal kudu/127.28.239.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.252.keytab.
I20260811 06:35:52.211202 29631 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJveEHt/build/asan/bin/kudu
/tmp/dist-test-taskJveEHt/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429755927508-29631-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429755927508-29631-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429755927508-29631-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429755927508-29631-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.252:34949
--webserver_interface=127.28.239.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.252.keytab
--principal=kudu/127.28.239.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.239.212:34597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.239.254:37761,127.28.239.253:37315,127.28.239.252:34949
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429755927508-29631-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429755927508-29631-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429755927508-29631-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 06:35:52.233037 803 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.239.252" port: 34949 } attempt: 1
I20260811 06:35:52.234357 807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:52.233870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.252:34949 (local address 127.0.0.1:45842)
0811 06:35:52.234079 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0811 06:35:52.234198 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.252:34949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260811 06:35:52.235371 807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:52.235027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.252:34949 (local address 127.0.0.1:45852)
0811 06:35:52.235189 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0811 06:35:52.235266 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.252:34949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260811 06:35:52.235585 803 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.239.252:34949: Network error: Client connection negotiation failed: client connection to 127.28.239.252:34949: connect: Connection refused (error 111)
I20260811 06:35:52.238399 689 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.239.253" port: 37315 } attempt: 5
Aug 11 06:35:52 dist-test-slave-gkw7 krb5kdc[571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430151, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.254@KRBTEST.COM for kudu/127.28.239.253@KRBTEST.COM
I20260811 06:35:52.253631 693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:52.239080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.253:37315 (local address 127.0.0.1:53578)
0811 06:35:52.239254 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0811 06:35:52.239276 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 06:35:52.239390 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:35:52.239774 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:35:52.239777 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:35:52.240156 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:35:52.240340 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:35:52.240349 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:35:52.241938 (+ 1589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:35:52.241942 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:35:52.243025 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:35:52.243038 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:35:52.244231 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:35:52.244236 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:35:52.244387 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:52.245022 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:35:52.245040 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:35:52.247456 (+ 2416us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:35:52.251358 (+ 3902us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:35:52.251362 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:35:52.251365 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:35:52.251727 (+ 362us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:35:52.252103 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:35:52.252106 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:35:52.252108 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:35:52.252259 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:35:52.252841 (+ 582us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:35:52.252844 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:35:52.252967 (+ 123us) client_negotiation.cc:769] Sending connection context
0811 06:35:52.253077 (+ 110us) client_negotiation.cc:240] Negotiation successful
0811 06:35:52.253324 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260811 06:35:52.253788 810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:52.239164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53578 (local address 127.28.239.253:37315)
0811 06:35:52.239369 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:52.239378 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:52.239403 (+ 25us) server_negotiation.cc:407] Connection header received
0811 06:35:52.239518 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:52.239526 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:52.239626 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:52.239738 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:52.240500 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:52.241802 (+ 1302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:52.243198 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:52.244054 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:52.244229 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:52.247667 (+ 3438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:35:52.247676 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:35:52.247684 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:35:52.247734 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:35:52.251226 (+ 3492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:35:52.251835 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:35:52.251841 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:35:52.251845 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:35:52.251989 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:35:52.252359 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:35:52.252364 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:35:52.252368 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:35:52.252722 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:35:52.252859 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 06:35:52.253218 (+ 359us) server_negotiation.cc:299] Negotiation successful
0811 06:35:52.253444 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 06:35:52.254788 689 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.239.252" port: 34949 } has no permanent_uuid. Determining permanent_uuid...
I20260811 06:35:52.255968 693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:52.255564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.252:34949 (local address 127.0.0.1:45860)
0811 06:35:52.255764 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0811 06:35:52.255845 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.252:34949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260811 06:35:52.256943 693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:52.256613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.252:34949 (local address 127.0.0.1:45866)
0811 06:35:52.256782 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0811 06:35:52.256843 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.252:34949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 06:35:52.257145 689 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.239.252:34949: Network error: Client connection negotiation failed: client connection to 127.28.239.252:34949: connect: Connection refused (error 111)
I20260811 06:35:52.303870 803 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.239.252" port: 34949 } attempt: 2
I20260811 06:35:52.305351 807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:52.304787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.252:34949 (local address 127.0.0.1:45872)
0811 06:35:52.305030 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0811 06:35:52.305161 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.252:34949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260811 06:35:52.306608 807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:52.306198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.252:34949 (local address 127.0.0.1:45882)
0811 06:35:52.306367 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0811 06:35:52.306456 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.252:34949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260811 06:35:52.306926 803 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.239.252:34949: Network error: Client connection negotiation failed: client connection to 127.28.239.252:34949: connect: Connection refused (error 111)
I20260811 06:35:52.308408 689 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.239.252" port: 34949 } attempt: 1
I20260811 06:35:52.309615 693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:52.309058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.252:34949 (local address 127.0.0.1:45896)
0811 06:35:52.309289 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0811 06:35:52.309391 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.252:34949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260811 06:35:52.310791 693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:52.310414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.252:34949 (local address 127.0.0.1:45910)
0811 06:35:52.310575 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0811 06:35:52.310660 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.252:34949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260811 06:35:52.311041 689 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.239.252:34949: Network error: Client connection negotiation failed: client connection to 127.28.239.252:34949: connect: Connection refused (error 111)
I20260811 06:35:52.379333 689 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.239.252" port: 34949 } attempt: 2
I20260811 06:35:52.380805 693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:52.380118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.252:34949 (local address 127.0.0.1:45926)
0811 06:35:52.380423 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0811 06:35:52.380550 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.252:34949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260811 06:35:52.382277 693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:52.381753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.252:34949 (local address 127.0.0.1:45940)
0811 06:35:52.381955 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0811 06:35:52.382084 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.252:34949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260811 06:35:52.382608 689 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.239.252:34949: Network error: Client connection negotiation failed: client connection to 127.28.239.252:34949: connect: Connection refused (error 111)
I20260811 06:35:52.398206 803 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.239.252" port: 34949 } attempt: 3
I20260811 06:35:52.399420 807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:52.398935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.252:34949 (local address 127.0.0.1:45942)
0811 06:35:52.399142 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0811 06:35:52.399251 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.252:34949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260811 06:35:52.400547 807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:52.400118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.252:34949 (local address 127.0.0.1:45944)
0811 06:35:52.400360 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0811 06:35:52.400433 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.252:34949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131}
W20260811 06:35:52.400828 803 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.239.252:34949: Network error: Client connection negotiation failed: client connection to 127.28.239.252:34949: connect: Connection refused (error 111)
I20260811 06:35:52.488871 689 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.239.252" port: 34949 } attempt: 3
I20260811 06:35:52.490190 693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:52.489599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.252:34949 (local address 127.0.0.1:45952)
0811 06:35:52.489839 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0811 06:35:52.489955 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.252:34949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260811 06:35:52.491400 693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:52.490920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.252:34949 (local address 127.0.0.1:45958)
0811 06:35:52.491141 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0811 06:35:52.491253 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.252:34949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
W20260811 06:35:52.491634 689 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.239.252:34949: Network error: Client connection negotiation failed: client connection to 127.28.239.252:34949: connect: Connection refused (error 111)
I20260811 06:35:52.544091 803 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.239.252" port: 34949 } attempt: 4
I20260811 06:35:52.545334 807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:52.544778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.252:34949 (local address 127.0.0.1:45972)
0811 06:35:52.545047 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0811 06:35:52.545146 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.252:34949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260811 06:35:52.546428 807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:52.546085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.252:34949 (local address 127.0.0.1:45974)
0811 06:35:52.546254 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0811 06:35:52.546322 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.252:34949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260811 06:35:52.546701 803 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.239.252:34949: Network error: Client connection negotiation failed: client connection to 127.28.239.252:34949: connect: Connection refused (error 111)
W20260811 06:35:52.576674 814 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 06:35:52.577149 814 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:35:52.577277 814 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:35:52.577414 814 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:35:52.585032 814 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:35:52.585131 814 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:35:52.585260 814 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:35:52.585340 814 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:35:52.585458 814 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:35:52.595733 814 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.239.212:34597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429755927508-29631-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429755927508-29631-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.28.239.254:37761,127.28.239.253:37315,127.28.239.252:34949
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429755927508-29631-0/krb5kdc/kudu_127.28.239.252.keytab
--principal=kudu/127.28.239.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.239.252:34949
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429755927508-29631-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.28.239.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429755927508-29631-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:07:47 UTC on ccycloud.quasar-kfibjw.root.comops.site
build id 4579725
ASAN enabled
I20260811 06:35:52.597149 814 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:35:52.598659 814 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:35:52.611328 830 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:35:52.611452 823 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:35:52.611485 820 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:35:52.612527 814 server_base.cc:1034] running on GCE node
I20260811 06:35:52.613296 814 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:35:52.614805 814 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:35:52.616045 814 hybrid_clock.cc:648] HybridClock initialized: now 1786430152615971 us; error 53 us; skew 500 ppm
I20260811 06:35:52.616396 814 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:35:52 dist-test-slave-gkw7 krb5kdc[571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430152, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:35:52.619639 814 init.cc:377] Logged in from keytab as kudu/127.28.239.252@KRBTEST.COM (short username kudu)
I20260811 06:35:52.621126 814 webserver.cc:460] Webserver started at http://127.28.239.252:34901/ using document root <none> and password file <none>
I20260811 06:35:52.621912 814 fs_manager.cc:359] Metadata directory not provided
I20260811 06:35:52.622004 814 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:35:52.622289 814 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:35:52.625265 814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429755927508-29631-0/minicluster-data/master-2/data/instance:
uuid: "72705286c06348e4bd17024f9bcbc661"
format_stamp: "Formatted at 2026-08-11 06:35:52 on dist-test-slave-gkw7"
server_key: "89aad71fbdb0211b228153080b2db0ea"
server_key_iv: "0ff1e10c91bae391b563c71c4c4d78f6"
server_key_version: "encryptionkey@0"
I20260811 06:35:52.626096 814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429755927508-29631-0/minicluster-data/master-2/wal/instance:
uuid: "72705286c06348e4bd17024f9bcbc661"
format_stamp: "Formatted at 2026-08-11 06:35:52 on dist-test-slave-gkw7"
server_key: "89aad71fbdb0211b228153080b2db0ea"
server_key_iv: "0ff1e10c91bae391b563c71c4c4d78f6"
server_key_version: "encryptionkey@0"
I20260811 06:35:52.631539 814 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 06:35:52.635169 838 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:35:52.636710 814 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 06:35:52.636925 814 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429755927508-29631-0/minicluster-data/master-2/data,/tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429755927508-29631-0/minicluster-data/master-2/wal
uuid: "72705286c06348e4bd17024f9bcbc661"
format_stamp: "Formatted at 2026-08-11 06:35:52 on dist-test-slave-gkw7"
server_key: "89aad71fbdb0211b228153080b2db0ea"
server_key_iv: "0ff1e10c91bae391b563c71c4c4d78f6"
server_key_version: "encryptionkey@0"
I20260811 06:35:52.637200 814 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429755927508-29631-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429755927508-29631-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429755927508-29631-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:35:52.655766 814 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:35:52.661116 814 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:35:52.661505 814 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:35:52.668891 689 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.239.252" port: 34949 } attempt: 4
I20260811 06:35:52.670290 693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:52.669775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.252:34949 (local address 127.0.0.1:45984)
0811 06:35:52.669975 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0811 06:35:52.670097 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.252:34949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260811 06:35:52.671499 693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:52.671127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.252:34949 (local address 127.0.0.1:45990)
0811 06:35:52.671295 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0811 06:35:52.671371 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.239.252:34949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260811 06:35:52.671763 689 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.239.252:34949: Network error: Client connection negotiation failed: client connection to 127.28.239.252:34949: connect: Connection refused (error 111)
I20260811 06:35:52.684976 814 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.252:34949
I20260811 06:35:52.684994 913 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.239.252:34949 every 8 connection(s)
I20260811 06:35:52.686530 814 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429755927508-29631-0/minicluster-data/master-2/data/info.pb
I20260811 06:35:52.689256 29631 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJveEHt/build/asan/bin/kudu as pid 814
I20260811 06:35:52.689466 29631 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJveEHt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786429755927508-29631-0/minicluster-data/master-2/wal/instance
I20260811 06:35:52.690047 29631 external_mini_cluster.cc:1428] Setting key a380fd35979a0b3108ab792221079ac0
I20260811 06:35:52.691183 915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:35:52.701009 915 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.239.254" port: 37761 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 06:35:52 dist-test-slave-gkw7 krb5kdc[571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430150, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.239.252@KRBTEST.COM
Aug 11 06:35:52 dist-test-slave-gkw7 krb5kdc[571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430152, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.252@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
I20260811 06:35:52.715387 919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:52.691908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46002 (local address 127.28.239.252:34949)
0811 06:35:52.692719 (+ 811us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:52.692736 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:52.692787 (+ 51us) server_negotiation.cc:407] Connection header received
0811 06:35:52.694827 (+ 2040us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:52.694857 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:52.695407 (+ 550us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:52.695882 (+ 475us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:52.697558 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:52.699617 (+ 2059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:52.701021 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:52.702141 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:52.702377 (+ 236us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:52.707170 (+ 4793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:35:52.707180 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:35:52.707193 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:35:52.707248 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:35:52.710879 (+ 3631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:35:52.711789 (+ 910us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:35:52.711795 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:35:52.711801 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:35:52.711987 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:35:52.712435 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:35:52.712438 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:35:52.712440 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:35:52.712950 (+ 510us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:35:52.713263 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0811 06:35:52.713829 (+ 566us) server_negotiation.cc:299] Negotiation successful
0811 06:35:52.714251 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":562,"thread_start_us":332,"threads_started":1}
I20260811 06:35:52.720705 921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:52.704156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52366 (local address 127.28.239.254:37761)
0811 06:35:52.704787 (+ 631us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:52.704795 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:52.705245 (+ 450us) server_negotiation.cc:407] Connection header received
0811 06:35:52.705456 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:52.705466 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:52.705593 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:52.705731 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:52.706887 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:52.708061 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:52.709809 (+ 1748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:52.710453 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:52.710594 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:52.714230 (+ 3636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:35:52.714236 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:35:52.714240 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:35:52.714275 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:35:52.717045 (+ 2770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:35:52.717890 (+ 845us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:35:52.717893 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:35:52.717896 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:35:52.718017 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:35:52.718943 (+ 926us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:35:52.718948 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:35:52.718951 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:35:52.719326 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:35:52.719474 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 06:35:52.720148 (+ 674us) server_negotiation.cc:299] Negotiation successful
0811 06:35:52.720353 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":486,"thread_start_us":325,"threads_started":1}
I20260811 06:35:52.720709 922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:52.704202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.254:37761 (local address 127.0.0.1:52366)
0811 06:35:52.705098 (+ 896us) negotiation.cc:107] Waiting for socket to connect
0811 06:35:52.705140 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 06:35:52.705321 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:35:52.705794 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:35:52.705801 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:35:52.706467 (+ 666us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:35:52.706727 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:35:52.706741 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:35:52.708255 (+ 1514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:35:52.708261 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:35:52.709647 (+ 1386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:35:52.709664 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:35:52.710649 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:35:52.710654 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:35:52.710799 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:52.711415 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:35:52.711440 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:35:52.713887 (+ 2447us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:35:52.717178 (+ 3291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:35:52.717191 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:35:52.717198 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:35:52.717721 (+ 523us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:35:52.718133 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:35:52.718138 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:35:52.718143 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:35:52.718269 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:35:52.719458 (+ 1189us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:35:52.719466 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:35:52.719701 (+ 235us) client_negotiation.cc:769] Sending connection context
0811 06:35:52.719925 (+ 224us) client_negotiation.cc:240] Negotiation successful
0811 06:35:52.720266 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":692,"thread_start_us":540,"threads_started":1}
I20260811 06:35:52.722673 915 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.239.253" port: 37315 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 06:35:52 dist-test-slave-gkw7 krb5kdc[571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430150, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.239.254@KRBTEST.COM
Aug 11 06:35:52 dist-test-slave-gkw7 krb5kdc[571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430152, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.252@KRBTEST.COM for kudu/127.28.239.253@KRBTEST.COM
I20260811 06:35:52.734403 923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:52.718474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52374 (local address 127.28.239.254:37761)
0811 06:35:52.719127 (+ 653us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:52.719132 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:52.719155 (+ 23us) server_negotiation.cc:407] Connection header received
0811 06:35:52.719199 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:52.719204 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:52.719275 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:52.719392 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:52.721014 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:52.722092 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:52.723697 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:52.724136 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:52.724282 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:52.728736 (+ 4454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:35:52.728741 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:35:52.728744 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:35:52.728776 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:35:52.731660 (+ 2884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:35:52.732305 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:35:52.732311 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:35:52.732315 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:35:52.732453 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:35:52.732828 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:35:52.732833 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:35:52.732838 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:35:52.733111 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:35:52.733406 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0811 06:35:52.733785 (+ 379us) server_negotiation.cc:299] Negotiation successful
0811 06:35:52.734012 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":499,"thread_start_us":299,"threads_started":1}
I20260811 06:35:52.737887 922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:52.723452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.253:37315 (local address 127.0.0.1:53582)
0811 06:35:52.723670 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0811 06:35:52.723693 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 06:35:52.723790 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:35:52.724110 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:35:52.724115 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:35:52.724529 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:35:52.724743 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:35:52.724758 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:35:52.726335 (+ 1577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:35:52.726340 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:35:52.727335 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:35:52.727347 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:35:52.728275 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:35:52.728281 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:35:52.728484 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:52.729091 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:35:52.729109 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:35:52.731505 (+ 2396us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:35:52.735123 (+ 3618us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:35:52.735127 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:35:52.735130 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:35:52.735613 (+ 483us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:35:52.736062 (+ 449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:35:52.736068 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:35:52.736072 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:35:52.736261 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:35:52.736825 (+ 564us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:35:52.736830 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:35:52.737011 (+ 181us) client_negotiation.cc:769] Sending connection context
0811 06:35:52.737166 (+ 155us) client_negotiation.cc:240] Negotiation successful
0811 06:35:52.737467 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72}
I20260811 06:35:52.738205 810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:52.723610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53582 (local address 127.28.239.253:37315)
0811 06:35:52.723815 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:52.723823 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:52.723845 (+ 22us) server_negotiation.cc:407] Connection header received
0811 06:35:52.723907 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:52.723921 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:52.723979 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:52.724086 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:52.724925 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:52.726133 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:52.727523 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:52.728066 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:52.728222 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:52.731728 (+ 3506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:35:52.731734 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:35:52.731737 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:35:52.731774 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:35:52.734977 (+ 3203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:35:52.735733 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:35:52.735738 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:35:52.735740 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:35:52.735917 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:35:52.736384 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:35:52.736389 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:35:52.736394 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:35:52.736696 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:35:52.736871 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0811 06:35:52.737292 (+ 421us) server_negotiation.cc:299] Negotiation successful
0811 06:35:52.737580 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 06:35:52.739158 915 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.239.252" port: 34949 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 06:35:52 dist-test-slave-gkw7 krb5kdc[571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430150, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.239.253@KRBTEST.COM
Aug 11 06:35:52 dist-test-slave-gkw7 krb5kdc[571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430152, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.252@KRBTEST.COM for kudu/127.28.239.252@KRBTEST.COM
I20260811 06:35:52.755329 924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:52.736298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53590 (local address 127.28.239.253:37315)
0811 06:35:52.736987 (+ 689us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:52.736991 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:52.737121 (+ 130us) server_negotiation.cc:407] Connection header received
0811 06:35:52.737276 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:52.737282 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:52.737354 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:52.737472 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:52.739035 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:52.740134 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:52.741805 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:52.742504 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:52.742653 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:52.748053 (+ 5400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:35:52.748058 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:35:52.748062 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:35:52.748091 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:35:52.751634 (+ 3543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:35:52.752540 (+ 906us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:35:52.752544 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:35:52.752546 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:35:52.752684 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:35:52.753242 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:35:52.753247 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:35:52.753249 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:35:52.753809 (+ 560us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:35:52.754192 (+ 383us) server_negotiation.cc:1035] Waiting for connection context
0811 06:35:52.754645 (+ 453us) server_negotiation.cc:299] Negotiation successful
0811 06:35:52.754882 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":580,"thread_start_us":368,"threads_started":1}
I20260811 06:35:52.755416 922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:52.739993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.252:34949 (local address 127.0.0.1:46012)
0811 06:35:52.740150 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0811 06:35:52.740171 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 06:35:52.740287 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:35:52.740630 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:35:52.740634 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:35:52.741010 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:35:52.741177 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:35:52.741186 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:35:52.742631 (+ 1445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:35:52.742637 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:35:52.743983 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:35:52.744003 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:35:52.744804 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:35:52.744811 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:35:52.745074 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:52.745715 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:35:52.745738 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:35:52.748245 (+ 2507us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:35:52.752042 (+ 3797us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:35:52.752048 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:35:52.752052 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:35:52.752642 (+ 590us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:35:52.753045 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:35:52.753050 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:35:52.753054 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:35:52.753270 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:35:52.754167 (+ 897us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:35:52.754173 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:35:52.754355 (+ 182us) client_negotiation.cc:769] Sending connection context
0811 06:35:52.754564 (+ 209us) client_negotiation.cc:240] Negotiation successful
0811 06:35:52.754934 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260811 06:35:52.755811 919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:52.739924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46012 (local address 127.28.239.252:34949)
0811 06:35:52.740114 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:52.740119 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:52.740237 (+ 118us) server_negotiation.cc:407] Connection header received
0811 06:35:52.740424 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:52.740429 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:52.740499 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:52.740596 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:52.741325 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:52.742477 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:52.744179 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:52.744620 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:52.744723 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:52.748432 (+ 3709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:35:52.748438 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:35:52.748443 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:35:52.748482 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:35:52.751893 (+ 3411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:35:52.752767 (+ 874us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:35:52.752774 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:35:52.752779 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:35:52.752922 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:35:52.753424 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:35:52.753431 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:35:52.753435 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:35:52.753943 (+ 508us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:35:52.754191 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0811 06:35:52.754674 (+ 483us) server_negotiation.cc:299] Negotiation successful
0811 06:35:52.755144 (+ 470us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":194,"server-negotiator.queue_time_us":64}
I20260811 06:35:52.761579 29631 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 06:35:52.762200 915 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72705286c06348e4bd17024f9bcbc661: Bootstrap starting.
I20260811 06:35:52.766026 915 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 72705286c06348e4bd17024f9bcbc661: Neither blocks nor log segments found. Creating new log.
I20260811 06:35:52.764055 650 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "90e689ca66554b29aea235643641e4ea"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52374
I20260811 06:35:52.767275 915 log.cc:824] T 00000000000000000000000000000000 P 72705286c06348e4bd17024f9bcbc661: Log is configured to *not* fsync() on all Append() calls
I20260811 06:35:52.770411 915 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72705286c06348e4bd17024f9bcbc661: No bootstrap required, opened a new log
I20260811 06:35:52.780025 915 raft_consensus.cc:348] T 00000000000000000000000000000000 P 72705286c06348e4bd17024f9bcbc661 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e689ca66554b29aea235643641e4ea" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 37761 } } peers { permanent_uuid: "dac31c807aa04f0ab3869b1f943ecb0b" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 37315 } } peers { permanent_uuid: "72705286c06348e4bd17024f9bcbc661" member_type: VOTER last_known_addr { host: "127.28.239.252" port: 34949 } }
I20260811 06:35:52.780441 915 raft_consensus.cc:729] T 00000000000000000000000000000000 P 72705286c06348e4bd17024f9bcbc661 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72705286c06348e4bd17024f9bcbc661, State: Initialized, Role: FOLLOWER
I20260811 06:35:52.780968 915 consensus_queue.cc:260] T 00000000000000000000000000000000 P 72705286c06348e4bd17024f9bcbc661 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e689ca66554b29aea235643641e4ea" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 37761 } } peers { permanent_uuid: "dac31c807aa04f0ab3869b1f943ecb0b" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 37315 } } peers { permanent_uuid: "72705286c06348e4bd17024f9bcbc661" member_type: VOTER last_known_addr { host: "127.28.239.252" port: 34949 } }
I20260811 06:35:52.782068 930 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72705286c06348e4bd17024f9bcbc661 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e689ca66554b29aea235643641e4ea" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 37761 } } peers { permanent_uuid: "dac31c807aa04f0ab3869b1f943ecb0b" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 37315 } } peers { permanent_uuid: "72705286c06348e4bd17024f9bcbc661" member_type: VOTER last_known_addr { host: "127.28.239.252" port: 34949 } } }
I20260811 06:35:52.782476 930 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72705286c06348e4bd17024f9bcbc661 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:35:52.783504 915 sys_catalog.cc:564] T 00000000000000000000000000000000 P 72705286c06348e4bd17024f9bcbc661 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 06:35:52.793879 945 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 72705286c06348e4bd17024f9bcbc661: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 06:35:52.794062 945 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 06:35:52.791728 814 master_runner.cc:186] Error getting master registration for 127.28.239.254:37761: OK, instance_id { permanent_uuid: "90e689ca66554b29aea235643641e4ea" instance_seqno: 1786430151522075 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 06:35:52.799937 814 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 06:35:52.845888 803 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.239.252" port: 34949 } attempt: 5
Aug 11 06:35:52 dist-test-slave-gkw7 krb5kdc[571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430152, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.253@KRBTEST.COM for kudu/127.28.239.252@KRBTEST.COM
I20260811 06:35:52.862748 919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:52.847081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46014 (local address 127.28.239.252:34949)
0811 06:35:52.847309 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:52.847313 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:52.847329 (+ 16us) server_negotiation.cc:407] Connection header received
0811 06:35:52.847384 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:52.847387 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:52.847445 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:52.847566 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:52.848590 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:52.849337 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:52.850585 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:52.851080 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:52.851228 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:52.855560 (+ 4332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:35:52.855567 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:35:52.855571 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:35:52.855615 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:35:52.859439 (+ 3824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:35:52.860377 (+ 938us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:35:52.860382 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:35:52.860386 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:35:52.860555 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:35:52.861024 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:35:52.861029 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:35:52.861033 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:35:52.861418 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:35:52.861618 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0811 06:35:52.862122 (+ 504us) server_negotiation.cc:299] Negotiation successful
0811 06:35:52.862353 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 06:35:52.862865 807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:52.846697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.252:34949 (local address 127.0.0.1:46014)
0811 06:35:52.846935 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0811 06:35:52.846958 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 06:35:52.847095 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:35:52.847614 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:35:52.847621 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:35:52.848167 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:35:52.848403 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:35:52.848415 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:35:52.849482 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:35:52.849488 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:35:52.850439 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:35:52.850451 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:35:52.851240 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:35:52.851245 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:35:52.851497 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:52.852227 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:35:52.852254 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:35:52.855342 (+ 3088us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:35:52.859608 (+ 4266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:35:52.859614 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:35:52.859619 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:35:52.860248 (+ 629us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:35:52.860686 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:35:52.860691 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:35:52.860694 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:35:52.860899 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:35:52.861597 (+ 698us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:35:52.861602 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:35:52.861804 (+ 202us) client_negotiation.cc:769] Sending connection context
0811 06:35:52.862005 (+ 201us) client_negotiation.cc:240] Negotiation successful
0811 06:35:52.862438 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82}
I20260811 06:35:52.867992 803 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dac31c807aa04f0ab3869b1f943ecb0b: Bootstrap starting.
I20260811 06:35:52.871294 803 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dac31c807aa04f0ab3869b1f943ecb0b: Neither blocks nor log segments found. Creating new log.
I20260811 06:35:52.872388 803 log.cc:824] T 00000000000000000000000000000000 P dac31c807aa04f0ab3869b1f943ecb0b: Log is configured to *not* fsync() on all Append() calls
I20260811 06:35:52.875250 803 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dac31c807aa04f0ab3869b1f943ecb0b: No bootstrap required, opened a new log
I20260811 06:35:52.883065 803 raft_consensus.cc:348] T 00000000000000000000000000000000 P dac31c807aa04f0ab3869b1f943ecb0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e689ca66554b29aea235643641e4ea" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 37761 } } peers { permanent_uuid: "dac31c807aa04f0ab3869b1f943ecb0b" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 37315 } } peers { permanent_uuid: "72705286c06348e4bd17024f9bcbc661" member_type: VOTER last_known_addr { host: "127.28.239.252" port: 34949 } }
I20260811 06:35:52.883540 803 raft_consensus.cc:729] T 00000000000000000000000000000000 P dac31c807aa04f0ab3869b1f943ecb0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dac31c807aa04f0ab3869b1f943ecb0b, State: Initialized, Role: FOLLOWER
I20260811 06:35:52.884132 803 consensus_queue.cc:260] T 00000000000000000000000000000000 P dac31c807aa04f0ab3869b1f943ecb0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e689ca66554b29aea235643641e4ea" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 37761 } } peers { permanent_uuid: "dac31c807aa04f0ab3869b1f943ecb0b" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 37315 } } peers { permanent_uuid: "72705286c06348e4bd17024f9bcbc661" member_type: VOTER last_known_addr { host: "127.28.239.252" port: 34949 } }
I20260811 06:35:52.885190 951 sys_catalog.cc:455] T 00000000000000000000000000000000 P dac31c807aa04f0ab3869b1f943ecb0b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e689ca66554b29aea235643641e4ea" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 37761 } } peers { permanent_uuid: "dac31c807aa04f0ab3869b1f943ecb0b" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 37315 } } peers { permanent_uuid: "72705286c06348e4bd17024f9bcbc661" member_type: VOTER last_known_addr { host: "127.28.239.252" port: 34949 } } }
I20260811 06:35:52.885665 951 sys_catalog.cc:458] T 00000000000000000000000000000000 P dac31c807aa04f0ab3869b1f943ecb0b [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:35:52.886672 803 sys_catalog.cc:564] T 00000000000000000000000000000000 P dac31c807aa04f0ab3869b1f943ecb0b [sys.catalog]: configured and running, proceeding with master startup.
W20260811 06:35:52.903775 967 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dac31c807aa04f0ab3869b1f943ecb0b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 06:35:52.903971 967 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 06:35:52.900792 702 master_runner.cc:186] Error getting master registration for 127.28.239.254:37761: OK, instance_id { permanent_uuid: "90e689ca66554b29aea235643641e4ea" instance_seqno: 1786430151522075 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
W20260811 06:35:52.914003 846 tablet.cc:2369] T 00000000000000000000000000000000 P 72705286c06348e4bd17024f9bcbc661: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:35:52.914099 702 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 06:35:52.949030 689 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.239.252" port: 34949 } attempt: 5
Aug 11 06:35:52 dist-test-slave-gkw7 krb5kdc[571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786430151, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.254@KRBTEST.COM for kudu/127.28.239.252@KRBTEST.COM
I20260811 06:35:52.968678 693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:52.949907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.239.252:34949 (local address 127.0.0.1:46026)
0811 06:35:52.950252 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0811 06:35:52.950274 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 06:35:52.950432 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:35:52.950996 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:35:52.951003 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:35:52.951595 (+ 592us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:35:52.951856 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:35:52.951870 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:35:52.953450 (+ 1580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:35:52.953457 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:35:52.955309 (+ 1852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:35:52.955328 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:35:52.956395 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:35:52.956403 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:35:52.956642 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:52.957572 (+ 930us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:35:52.957602 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:35:52.960876 (+ 3274us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:35:52.965579 (+ 4703us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:35:52.965587 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:35:52.965591 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:35:52.966183 (+ 592us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:35:52.966646 (+ 463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:35:52.966651 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:35:52.966656 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:35:52.966854 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:35:52.967490 (+ 636us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:35:52.967498 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:35:52.967709 (+ 211us) client_negotiation.cc:769] Sending connection context
0811 06:35:52.967879 (+ 170us) client_negotiation.cc:240] Negotiation successful
0811 06:35:52.968220 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205}
I20260811 06:35:52.968688 919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:52.949987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46026 (local address 127.28.239.252:34949)
0811 06:35:52.950205 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:52.950211 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:52.950377 (+ 166us) server_negotiation.cc:407] Connection header received
0811 06:35:52.950651 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:52.950657 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:52.950792 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:52.950911 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:52.952045 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:52.953281 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:52.955468 (+ 2187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:52.956187 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:52.956334 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:52.961087 (+ 4753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:35:52.961094 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:35:52.961099 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:35:52.961143 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:35:52.965416 (+ 4273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:35:52.966342 (+ 926us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:35:52.966348 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:35:52.966352 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:35:52.966534 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:35:52.966995 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:35:52.967000 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:35:52.967004 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:35:52.967375 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:35:52.967530 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 06:35:52.968033 (+ 503us) server_negotiation.cc:299] Negotiation successful
0811 06:35:52.968304 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 06:35:52.975672 689 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90e689ca66554b29aea235643641e4ea: Bootstrap starting.
I20260811 06:35:52.980087 689 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 90e689ca66554b29aea235643641e4ea: Neither blocks nor log segments found. Creating new log.
I20260811 06:35:52.981498 689 log.cc:824] T 00000000000000000000000000000000 P 90e689ca66554b29aea235643641e4ea: Log is configured to *not* fsync() on all Append() calls
I20260811 06:35:52.986213 689 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90e689ca66554b29aea235643641e4ea: No bootstrap required, opened a new log
I20260811 06:35:52.994787 689 raft_consensus.cc:348] T 00000000000000000000000000000000 P 90e689ca66554b29aea235643641e4ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e689ca66554b29aea235643641e4ea" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 37761 } } peers { permanent_uuid: "dac31c807aa04f0ab3869b1f943ecb0b" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 37315 } } peers { permanent_uuid: "72705286c06348e4bd17024f9bcbc661" member_type: VOTER last_known_addr { host: "127.28.239.252" port: 34949 } }
I20260811 06:35:52.995227 689 raft_consensus.cc:729] T 00000000000000000000000000000000 P 90e689ca66554b29aea235643641e4ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90e689ca66554b29aea235643641e4ea, State: Initialized, Role: FOLLOWER
I20260811 06:35:52.995774 689 consensus_queue.cc:260] T 00000000000000000000000000000000 P 90e689ca66554b29aea235643641e4ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e689ca66554b29aea235643641e4ea" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 37761 } } peers { permanent_uuid: "dac31c807aa04f0ab3869b1f943ecb0b" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 37315 } } peers { permanent_uuid: "72705286c06348e4bd17024f9bcbc661" member_type: VOTER last_known_addr { host: "127.28.239.252" port: 34949 } }
I20260811 06:35:52.997219 971 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90e689ca66554b29aea235643641e4ea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e689ca66554b29aea235643641e4ea" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 37761 } } peers { permanent_uuid: "dac31c807aa04f0ab3869b1f943ecb0b" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 37315 } } peers { permanent_uuid: "72705286c06348e4bd17024f9bcbc661" member_type: VOTER last_known_addr { host: "127.28.239.252" port: 34949 } } }
I20260811 06:35:52.998075 971 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90e689ca66554b29aea235643641e4ea [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:35:52.998342 689 sys_catalog.cc:564] T 00000000000000000000000000000000 P 90e689ca66554b29aea235643641e4ea [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:35:52.998448 650 raft_consensus.cc:482] T 00000000000000000000000000000000 P 90e689ca66554b29aea235643641e4ea [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 06:35:52.998642 650 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 90e689ca66554b29aea235643641e4ea [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:35:53.002491 650 raft_consensus.cc:504] T 00000000000000000000000000000000 P 90e689ca66554b29aea235643641e4ea [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e689ca66554b29aea235643641e4ea" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 37761 } } peers { permanent_uuid: "dac31c807aa04f0ab3869b1f943ecb0b" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 37315 } } peers { permanent_uuid: "72705286c06348e4bd17024f9bcbc661" member_type: VOTER last_known_addr { host: "127.28.239.252" port: 34949 } }
I20260811 06:35:53.005381 767 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "90e689ca66554b29aea235643641e4ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "dac31c807aa04f0ab3869b1f943ecb0b"
I20260811 06:35:53.005954 767 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dac31c807aa04f0ab3869b1f943ecb0b [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:35:53.010149 767 raft_consensus.cc:2454] T 00000000000000000000000000000000 P dac31c807aa04f0ab3869b1f943ecb0b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e689ca66554b29aea235643641e4ea in term 1.
W20260811 06:35:53.010850 622 tablet.cc:2369] T 00000000000000000000000000000000 P 90e689ca66554b29aea235643641e4ea: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:35:53.011067 618 leader_election.cc:302] T 00000000000000000000000000000000 P 90e689ca66554b29aea235643641e4ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e689ca66554b29aea235643641e4ea, dac31c807aa04f0ab3869b1f943ecb0b; no voters:
I20260811 06:35:53.014689 878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "90e689ca66554b29aea235643641e4ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "72705286c06348e4bd17024f9bcbc661"
I20260811 06:35:53.015302 878 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 72705286c06348e4bd17024f9bcbc661 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:35:53.018090 977 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 90e689ca66554b29aea235643641e4ea [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:35:53.019373 878 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 72705286c06348e4bd17024f9bcbc661 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e689ca66554b29aea235643641e4ea in term 1.
I20260811 06:35:53.020428 650 leader_election.cc:288] T 00000000000000000000000000000000 P 90e689ca66554b29aea235643641e4ea [CANDIDATE]: Term 1 election: Requested vote from peers dac31c807aa04f0ab3869b1f943ecb0b (127.28.239.253:37315), 72705286c06348e4bd17024f9bcbc661 (127.28.239.252:34949)
I20260811 06:35:53.029016 977 raft_consensus.cc:686] T 00000000000000000000000000000000 P 90e689ca66554b29aea235643641e4ea [term 1 LEADER]: Becoming Leader. State: Replica: 90e689ca66554b29aea235643641e4ea, State: Running, Role: LEADER
I20260811 06:35:53.030041 977 consensus_queue.cc:237] T 00000000000000000000000000000000 P 90e689ca66554b29aea235643641e4ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e689ca66554b29aea235643641e4ea" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 37761 } } peers { permanent_uuid: "dac31c807aa04f0ab3869b1f943ecb0b" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 37315 } } peers { permanent_uuid: "72705286c06348e4bd17024f9bcbc661" member_type: VOTER last_known_addr { host: "127.28.239.252" port: 34949 } }
I20260811 06:35:53.038797 971 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90e689ca66554b29aea235643641e4ea [sys.catalog]: SysCatalogTable state changed. Reason: New leader 90e689ca66554b29aea235643641e4ea. Latest consensus state: current_term: 1 leader_uuid: "90e689ca66554b29aea235643641e4ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e689ca66554b29aea235643641e4ea" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 37761 } } peers { permanent_uuid: "dac31c807aa04f0ab3869b1f943ecb0b" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 37315 } } peers { permanent_uuid: "72705286c06348e4bd17024f9bcbc661" member_type: VOTER last_known_addr { host: "127.28.239.252" port: 34949 } } }
I20260811 06:35:53.039292 971 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90e689ca66554b29aea235643641e4ea [sys.catalog]: This master's current role is: LEADER
I20260811 06:35:53.047817 992 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:35:53.061173 992 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:35:53.082125 1009 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
W20260811 06:35:53.106148 740 tablet.cc:2369] T 00000000000000000000000000000000 P dac31c807aa04f0ab3869b1f943ecb0b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:35:53.110353 924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:53.078927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53606 (local address 127.28.239.253:37315)
0811 06:35:53.079168 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:53.079176 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:53.082353 (+ 3177us) server_negotiation.cc:407] Connection header received
0811 06:35:53.084518 (+ 2165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:53.084525 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:53.084619 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:53.084701 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:53.093792 (+ 9091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:53.095020 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:53.096952 (+ 1932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:53.097680 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:53.097796 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:53.101749 (+ 3953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:35:53.101757 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:35:53.101762 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:35:53.101831 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:35:53.105776 (+ 3945us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:35:53.107804 (+ 2028us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:35:53.107811 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:35:53.107815 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:35:53.107980 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:35:53.108474 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:35:53.108482 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:35:53.108490 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:35:53.108864 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:35:53.109103 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0811 06:35:53.109700 (+ 597us) server_negotiation.cc:299] Negotiation successful
0811 06:35:53.109972 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 06:35:53.120008 878 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 72705286c06348e4bd17024f9bcbc661 [term 1 FOLLOWER]: Refusing update from remote peer 90e689ca66554b29aea235643641e4ea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:35:53.121474 767 raft_consensus.cc:1261] T 00000000000000000000000000000000 P dac31c807aa04f0ab3869b1f943ecb0b [term 1 FOLLOWER]: Refusing update from remote peer 90e689ca66554b29aea235643641e4ea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:35:53.122968 977 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 90e689ca66554b29aea235643641e4ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "dac31c807aa04f0ab3869b1f943ecb0b" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 37315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:35:53.123749 977 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 90e689ca66554b29aea235643641e4ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "72705286c06348e4bd17024f9bcbc661" member_type: VOTER last_known_addr { host: "127.28.239.252" port: 34949 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:35:53.145128 923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:53.065457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52382 (local address 127.28.239.254:37761)
0811 06:35:53.065703 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:53.065710 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:53.072528 (+ 6818us) server_negotiation.cc:407] Connection header received
0811 06:35:53.079516 (+ 6988us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:53.079524 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:53.079639 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:53.079734 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:53.088856 (+ 9122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:53.090242 (+ 1386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:53.106336 (+ 16094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:53.107118 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:53.107237 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:53.116282 (+ 9045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:35:53.116290 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:35:53.116294 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:35:53.116346 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:35:53.126581 (+ 10235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:35:53.138841 (+ 12260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:35:53.138851 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:35:53.138857 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:35:53.139106 (+ 249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:35:53.139762 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:35:53.139772 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:35:53.139778 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:35:53.140370 (+ 592us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:35:53.140663 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0811 06:35:53.144184 (+ 3521us) server_negotiation.cc:299] Negotiation successful
0811 06:35:53.144556 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 06:35:53.157739 919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:53.070245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46034 (local address 127.28.239.252:34949)
0811 06:35:53.070449 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:53.070456 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:53.078229 (+ 7773us) server_negotiation.cc:407] Connection header received
0811 06:35:53.082835 (+ 4606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:53.082844 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:53.082952 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:53.083062 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:53.101692 (+ 18630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:53.103008 (+ 1316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:53.108528 (+ 5520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:53.109431 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:53.109571 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:53.141472 (+ 31901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:35:53.141484 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:35:53.141492 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:35:53.141643 (+ 151us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:35:53.150579 (+ 8936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:35:53.151711 (+ 1132us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:35:53.151717 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:35:53.151721 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:35:53.151891 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:35:53.152431 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:35:53.152437 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:35:53.152441 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:35:53.152900 (+ 459us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:35:53.153180 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0811 06:35:53.156949 (+ 3769us) server_negotiation.cc:299] Negotiation successful
0811 06:35:53.157276 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 06:35:53.169425 930 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72705286c06348e4bd17024f9bcbc661 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 90e689ca66554b29aea235643641e4ea. Latest consensus state: current_term: 1 leader_uuid: "90e689ca66554b29aea235643641e4ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e689ca66554b29aea235643641e4ea" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 37761 } } peers { permanent_uuid: "dac31c807aa04f0ab3869b1f943ecb0b" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 37315 } } peers { permanent_uuid: "72705286c06348e4bd17024f9bcbc661" member_type: VOTER last_known_addr { host: "127.28.239.252" port: 34949 } } }
I20260811 06:35:53.169909 930 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72705286c06348e4bd17024f9bcbc661 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:35:53.178575 1017 mvcc.cc:205] Tried to move back new op lower bound from 7317217907164164096 to 7317217906819809280. Current Snapshot: MvccSnapshot[applied={T|T < 7317217907164164096}]
I20260811 06:35:53.186007 951 sys_catalog.cc:455] T 00000000000000000000000000000000 P dac31c807aa04f0ab3869b1f943ecb0b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 90e689ca66554b29aea235643641e4ea. Latest consensus state: current_term: 1 leader_uuid: "90e689ca66554b29aea235643641e4ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e689ca66554b29aea235643641e4ea" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 37761 } } peers { permanent_uuid: "dac31c807aa04f0ab3869b1f943ecb0b" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 37315 } } peers { permanent_uuid: "72705286c06348e4bd17024f9bcbc661" member_type: VOTER last_known_addr { host: "127.28.239.252" port: 34949 } } }
I20260811 06:35:53.186388 951 sys_catalog.cc:458] T 00000000000000000000000000000000 P dac31c807aa04f0ab3869b1f943ecb0b [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:35:53.188326 978 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90e689ca66554b29aea235643641e4ea [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "90e689ca66554b29aea235643641e4ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e689ca66554b29aea235643641e4ea" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 37761 } } peers { permanent_uuid: "dac31c807aa04f0ab3869b1f943ecb0b" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 37315 } } peers { permanent_uuid: "72705286c06348e4bd17024f9bcbc661" member_type: VOTER last_known_addr { host: "127.28.239.252" port: 34949 } } }
I20260811 06:35:53.188944 992 catalog_manager.cc:1347] Generated new cluster ID: 2d0c5c08fb46495ea53c80fd45936540
I20260811 06:35:53.189029 978 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90e689ca66554b29aea235643641e4ea [sys.catalog]: This master's current role is: LEADER
I20260811 06:35:53.189074 992 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:35:53.189630 930 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72705286c06348e4bd17024f9bcbc661 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "90e689ca66554b29aea235643641e4ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e689ca66554b29aea235643641e4ea" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 37761 } } peers { permanent_uuid: "dac31c807aa04f0ab3869b1f943ecb0b" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 37315 } } peers { permanent_uuid: "72705286c06348e4bd17024f9bcbc661" member_type: VOTER last_known_addr { host: "127.28.239.252" port: 34949 } } }
I20260811 06:35:53.189930 930 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72705286c06348e4bd17024f9bcbc661 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:35:53.200337 977 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90e689ca66554b29aea235643641e4ea [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "90e689ca66554b29aea235643641e4ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e689ca66554b29aea235643641e4ea" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 37761 } } peers { permanent_uuid: "dac31c807aa04f0ab3869b1f943ecb0b" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 37315 } } peers { permanent_uuid: "72705286c06348e4bd17024f9bcbc661" member_type: VOTER last_known_addr { host: "127.28.239.252" port: 34949 } } }
I20260811 06:35:53.200732 977 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90e689ca66554b29aea235643641e4ea [sys.catalog]: This master's current role is: LEADER
I20260811 06:35:53.202052 1015 mvcc.cc:205] Tried to move back new op lower bound from 7317217907164164096 to 7317217906819809280. Current Snapshot: MvccSnapshot[applied={T|T < 7317217907164164096}]
I20260811 06:35:53.206751 951 sys_catalog.cc:455] T 00000000000000000000000000000000 P dac31c807aa04f0ab3869b1f943ecb0b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "90e689ca66554b29aea235643641e4ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e689ca66554b29aea235643641e4ea" member_type: VOTER last_known_addr { host: "127.28.239.254" port: 37761 } } peers { permanent_uuid: "dac31c807aa04f0ab3869b1f943ecb0b" member_type: VOTER last_known_addr { host: "127.28.239.253" port: 37315 } } peers { permanent_uuid: "72705286c06348e4bd17024f9bcbc661" member_type: VOTER last_known_addr { host: "127.28.239.252" port: 34949 } } }
I20260811 06:35:53.207057 951 sys_catalog.cc:458] T 00000000000000000000000000000000 P dac31c807aa04f0ab3869b1f943ecb0b [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:35:53.233636 992 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:35:53.235867 992 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:35:53.258446 992 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 90e689ca66554b29aea235643641e4ea: Generated new TSK 0
I20260811 06:35:53.259734 992 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 06:35:53.428193 923 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:35:53.416881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52384 (local address 127.28.239.254:37761)
0811 06:35:53.417063 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:53.417070 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:53.423755 (+ 6685us) server_negotiation.cc:407] Connection header received
0811 06:35:53.424354 (+ 599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:53.424362 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:53.424494 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:53.424584 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:53.427729 (+ 3145us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52384: BlockingRecv error: recv got EOF from 127.0.0.1:52384 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260811 06:35:53.430979 919 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:35:53.416497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46044 (local address 127.28.239.252:34949)
0811 06:35:53.416711 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:53.416719 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:53.429063 (+ 12344us) server_negotiation.cc:407] Connection header received
0811 06:35:53.429353 (+ 290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:53.429359 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:53.429445 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:53.429571 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:53.430701 (+ 1130us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46044: BlockingRecv error: recv got EOF from 127.0.0.1:46044 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
W20260811 06:35:53.437377 924 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:35:53.434525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53608 (local address 127.28.239.253:37315)
0811 06:35:53.434754 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:53.434763 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:53.435300 (+ 537us) server_negotiation.cc:407] Connection header received
0811 06:35:53.435592 (+ 292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:53.435598 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:53.435718 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:53.435805 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:53.437106 (+ 1301us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53608: BlockingRecv error: recv got EOF from 127.0.0.1:53608 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260811 06:35:53.489559 924 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:35:53.475491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53618 (local address 127.28.239.253:37315)
0811 06:35:53.475697 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:53.475704 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:53.487059 (+ 11355us) server_negotiation.cc:407] Connection header received
0811 06:35:53.487718 (+ 659us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:53.487725 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:53.487898 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:53.487995 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:53.489282 (+ 1287us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53618: BlockingRecv error: recv got EOF from 127.0.0.1:53618 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
W20260811 06:35:53.526067 924 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:35:53.523828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53622 (local address 127.28.239.253:37315)
0811 06:35:53.524025 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:53.524031 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:53.524055 (+ 24us) server_negotiation.cc:407] Connection header received
0811 06:35:53.524354 (+ 299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:53.524363 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:53.524468 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:53.524557 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:53.525812 (+ 1255us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53622: BlockingRecv error: recv got EOF from 127.0.0.1:53622 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260811 06:35:53.566293 924 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:35:53.560603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53630 (local address 127.28.239.253:37315)
0811 06:35:53.560810 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:53.560817 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:53.564054 (+ 3237us) server_negotiation.cc:407] Connection header received
0811 06:35:53.564483 (+ 429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:53.564491 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:53.564629 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:53.564715 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:53.566000 (+ 1285us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53630: BlockingRecv error: recv got EOF from 127.0.0.1:53630 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
W20260811 06:35:53.577378 924 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:35:53.575137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53634 (local address 127.28.239.253:37315)
0811 06:35:53.575402 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:53.575410 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:53.575760 (+ 350us) server_negotiation.cc:407] Connection header received
0811 06:35:53.576017 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:53.576022 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:53.576113 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:53.576238 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:53.577065 (+ 827us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53634: BlockingRecv error: recv got EOF from 127.0.0.1:53634 (error 108)
Metrics: {"server-negotiator.queue_time_us":114}
W20260811 06:35:53.592865 924 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:35:53.589930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53636 (local address 127.28.239.253:37315)
0811 06:35:53.590413 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:53.590421 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:53.591108 (+ 687us) server_negotiation.cc:407] Connection header received
0811 06:35:53.591403 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:53.591407 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:53.591500 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:53.591598 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:53.592608 (+ 1010us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53636: BlockingRecv error: recv got EOF from 127.0.0.1:53636 (error 108)
Metrics: {"server-negotiator.queue_time_us":286}
W20260811 06:35:53.616127 924 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:35:53.613720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53646 (local address 127.28.239.253:37315)
0811 06:35:53.614000 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:53.614007 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:53.614347 (+ 340us) server_negotiation.cc:407] Connection header received
0811 06:35:53.614536 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:53.614542 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:53.614656 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:53.614801 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:53.615788 (+ 987us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53646: BlockingRecv error: recv got EOF from 127.0.0.1:53646 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
W20260811 06:35:53.655521 924 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:35:53.653046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53652 (local address 127.28.239.253:37315)
0811 06:35:53.653317 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:53.653324 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:53.653888 (+ 564us) server_negotiation.cc:407] Connection header received
0811 06:35:53.654156 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:53.654161 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:53.654245 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:53.654370 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:53.655240 (+ 870us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53652: BlockingRecv error: recv got EOF from 127.0.0.1:53652 (error 108)
Metrics: {"server-negotiator.queue_time_us":132}
W20260811 06:35:53.726130 924 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:35:53.723787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53658 (local address 127.28.239.253:37315)
0811 06:35:53.724091 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:53.724100 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:53.724444 (+ 344us) server_negotiation.cc:407] Connection header received
0811 06:35:53.724750 (+ 306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:53.724755 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:53.724850 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:53.724976 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:53.725917 (+ 941us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53658: BlockingRecv error: recv got EOF from 127.0.0.1:53658 (error 108)
Metrics: {"server-negotiator.queue_time_us":118}
I20260811 06:35:53.796136 945 catalog_manager.cc:1259] Loaded cluster ID: 2d0c5c08fb46495ea53c80fd45936540
I20260811 06:35:53.796288 945 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 72705286c06348e4bd17024f9bcbc661: loading cluster ID for follower catalog manager: success
I20260811 06:35:53.799141 945 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 72705286c06348e4bd17024f9bcbc661: acquiring CA information for follower catalog manager: success
I20260811 06:35:53.801170 945 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 72705286c06348e4bd17024f9bcbc661: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260811 06:35:53.860298 924 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:35:53.858000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53666 (local address 127.28.239.253:37315)
0811 06:35:53.858388 (+ 388us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:53.858395 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:53.858826 (+ 431us) server_negotiation.cc:407] Connection header received
0811 06:35:53.859083 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:53.859088 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:53.859165 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:53.859292 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:35:53.860100 (+ 808us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53666: BlockingRecv error: recv got EOF from 127.0.0.1:53666 (error 108)
Metrics: {"server-negotiator.queue_time_us":190}
I20260811 06:35:53.906263 967 catalog_manager.cc:1259] Loaded cluster ID: 2d0c5c08fb46495ea53c80fd45936540
I20260811 06:35:53.906376 967 catalog_manager.cc:1552] T 00000000000000000000000000000000 P dac31c807aa04f0ab3869b1f943ecb0b: loading cluster ID for follower catalog manager: success
I20260811 06:35:53.909092 967 catalog_manager.cc:1574] T 00000000000000000000000000000000 P dac31c807aa04f0ab3869b1f943ecb0b: acquiring CA information for follower catalog manager: success
I20260811 06:35:53.911139 967 catalog_manager.cc:1602] T 00000000000000000000000000000000 P dac31c807aa04f0ab3869b1f943ecb0b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 06:35:54.127337 924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:54.120498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53676 (local address 127.28.239.253:37315)
0811 06:35:54.120695 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:54.120700 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:54.120906 (+ 206us) server_negotiation.cc:407] Connection header received
0811 06:35:54.121112 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:54.121117 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:54.121351 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:54.121499 (+ 148us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:35:54.122013 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:54.123177 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:54.124579 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:54.125267 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:54.125386 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:54.125927 (+ 541us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:35:54.126666 (+ 739us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:35:54.126843 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0811 06:35:54.126904 (+ 61us) server_negotiation.cc:299] Negotiation successful
0811 06:35:54.126993 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 06:35:54.142683 1142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:54.135016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46060 (local address 127.28.239.252:34949)
0811 06:35:54.136724 (+ 1708us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:54.136729 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:54.136747 (+ 18us) server_negotiation.cc:407] Connection header received
0811 06:35:54.136846 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:54.136850 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:54.137004 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:54.137158 (+ 154us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:35:54.137617 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:54.138516 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:54.140165 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:54.140580 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:54.140720 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:54.141327 (+ 607us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:35:54.142036 (+ 709us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:35:54.142212 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0811 06:35:54.142269 (+ 57us) server_negotiation.cc:299] Negotiation successful
0811 06:35:54.142361 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1603,"thread_start_us":1493,"threads_started":1}
I20260811 06:35:54.160490 1142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:54.148792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46062 (local address 127.28.239.252:34949)
0811 06:35:54.149021 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:54.149030 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:54.154338 (+ 5308us) server_negotiation.cc:407] Connection header received
0811 06:35:54.154562 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:54.154571 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:54.154701 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:54.154843 (+ 142us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:35:54.155436 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:54.156520 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:54.158689 (+ 2169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:54.159113 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:54.159258 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:54.159703 (+ 445us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:35:54.159813 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:35:54.159915 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 06:35:54.160062 (+ 147us) server_negotiation.cc:299] Negotiation successful
0811 06:35:54.160145 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 06:35:54.161592 924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:54.149692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53688 (local address 127.28.239.253:37315)
0811 06:35:54.149964 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:54.149974 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:54.155181 (+ 5207us) server_negotiation.cc:407] Connection header received
0811 06:35:54.155358 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:54.155367 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:54.155513 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:54.155627 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:35:54.156262 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:54.157301 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:54.159311 (+ 2010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:54.159997 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:54.160107 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:54.160768 (+ 661us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:35:54.160909 (+ 141us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:35:54.160970 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 06:35:54.161203 (+ 233us) server_negotiation.cc:299] Negotiation successful
0811 06:35:54.161302 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260811 06:35:54.163067 1154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:35:54.148492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52386 (local address 127.28.239.254:37761)
0811 06:35:54.155891 (+ 7399us) server_negotiation.cc:206] Beginning negotiation
0811 06:35:54.155897 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:35:54.155916 (+ 19us) server_negotiation.cc:407] Connection header received
0811 06:35:54.156034 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:35:54.156040 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:35:54.156264 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:35:54.156445 (+ 181us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:35:54.157087 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:54.158289 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:54.160553 (+ 2264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:35:54.161200 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:35:54.161335 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:35:54.161974 (+ 639us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:35:54.162424 (+ 450us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:35:54.162560 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 06:35:54.162671 (+ 111us) server_negotiation.cc:299] Negotiation successful
0811 06:35:54.162759 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7273,"thread_start_us":7160,"threads_started":1}
I20260811 06:35:54.168354 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 588
I20260811 06:35:54.169435 683 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:35:54.285930 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 588
W20260811 06:35:54.316499 732 connection.cc:535] server connection from 127.0.0.1:53578 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 06:35:54.317184 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 702
I20260811 06:35:54.318444 797 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:35:54.432808 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 702
I20260811 06:35:54.464612 29631 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJveEHt/build/asan/bin/kudu pid 814
I20260811 06:35:54.465972 906 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:35:54.579313 29631 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJveEHt/build/asan/bin/kudu with pid 814
2026-08-11T06:35:54Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (5348 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (10857 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (398588 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST